00010000:00010000:2.0F:1769076241.199879:0:6173:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0F:1769076242.223991:0:6653:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1769076242.543950:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) b0abecde-06ba-4498-83fd-7513d38e1d30->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1769076242.543965:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1769076242.543973:0:37:0:(import.c:65:import_set_state_nolock()) ffff895e4be1a800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1769076242.543988:0:37:0:(import.c:509:import_select_connection()) MGC192.168.204.117@tcp: connect to NID 192.168.204.117@tcp last attempt 1925 00000100:00080000:1.0:1769076242.544002:0:37:0:(import.c:588:import_select_connection()) MGC192.168.204.117@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1769076242.544012:0:37:0:(import.c:623:import_select_connection()) MGC192.168.204.117@tcp: import ffff895e4be1a800 using connection 192.168.204.117@tcp/192.168.204.117@tcp 00000100:00100000:1.0:1769076242.544083:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff895e4713d880 x1855008961347712/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1769076242.544144:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 73a2201c-08e3-493e-ad2e-98b4627bffd5->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1769076242.544150:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1769076242.544152:0:37:0:(import.c:65:import_set_state_nolock()) ffff895e42e8c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1769076242.544158:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: connect to NID 192.168.204.117@tcp last attempt 1925 00000100:00080000:1.0:1769076242.544163:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1769076242.544168:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: import ffff895e42e8c000 using connection 192.168.204.117@tcp/192.168.204.117@tcp 00000008:00100000:1.0:1769076242.544221:0:37:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681347 ocd_grant: 2097152, lost: 0. 00000100:00100000:1.0:1769076242.544243:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff895e4713ed80 x1855008961347840/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1769076242.544259:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 73a2201c-08e3-493e-ad2e-98b4627bffd5->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1769076242.544264:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1769076242.544268:0:37:0:(import.c:65:import_set_state_nolock()) ffff895e42e89800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1769076242.544273:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: connect to NID 192.168.204.117@tcp last attempt 1925 00000100:00080000:1.0:1769076242.544280:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1769076242.544287:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: import ffff895e42e89800 using connection 192.168.204.117@tcp/192.168.204.117@tcp 00000008:00100000:1.0:1769076242.544294:0:37:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681347 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0F:1769076242.544305:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e4713d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:b0abecde-06ba-4498-83fd-7513d38e1d30:2402:1855008961347712:192.168.204.117@tcp:250:kworker.0 00000100:00100000:1.0:1769076242.544345:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff895e4713e680 x1855008961347968/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:3.0:1769076242.544431:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp 00000100:00100000:3.0:1769076242.544508:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769076242/real 1769076242] req@ffff895e4713d880 x1855008961347712/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 1 dl 1769076267 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1769076242.544545:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff895e4713d880 x1855008961347712/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 1 dl 1769076267 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1769076242.544565:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e4be1a800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1769076242.544572:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.117@tcp failed (-110) 00000100:00100000:3.0:1769076242.544579:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e4713d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:b0abecde-06ba-4498-83fd-7513d38e1d30:2402:1855008961347712:192.168.204.117@tcp:250:kworker.0 00000100:00100000:3.0:1769076242.544593:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e4713ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961347840:192.168.204.117@tcp:38:kworker.0 00000800:00000100:3.0:1769076242.544651:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp 00000100:00100000:3.0:1769076242.544681:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769076242/real 1769076242] req@ffff895e4713ed80 x1855008961347840/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769076267 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1769076242.544717:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff895e4713ed80 x1855008961347840/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769076267 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1769076242.544795:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e42e8c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1769076242.544802:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.117@tcp failed (-110) 00000100:00100000:3.0:1769076242.544809:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e4713ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961347840:192.168.204.117@tcp:38:kworker.0 00000100:00100000:3.0:1769076242.544855:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e4713e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961347968:192.168.204.117@tcp:38:kworker.0 00000800:00000100:3.0:1769076242.544897:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp 00000100:00100000:3.0:1769076242.545586:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769076242/real 1769076242] req@ffff895e4713e680 x1855008961347968/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769076267 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1769076242.545639:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff895e4713e680 x1855008961347968/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769076267 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1769076242.545656:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e42e89800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1769076242.545662:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.204.117@tcp failed (-110) 00000100:00100000:3.0:1769076242.545669:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e4713e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961347968:192.168.204.117@tcp:38:kworker.0 00010000:00010000:1.0:1769076243.248032:0:6172:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0F:1769076244.271936:0:6173:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769076245.296077:0:6653:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769076246.319958:0:6172:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1769076247.344034:0:6173:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1769076247.437032:0:26836:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-sec test 22: test nodemap mapped_trusted_admin fileops ========================================================== 05:04:06 (1769076246) 00000100:00080000:0.0:1769076247.663942:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) b0abecde-06ba-4498-83fd-7513d38e1d30->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1769076247.663985:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1769076247.663992:0:8:0:(import.c:65:import_set_state_nolock()) ffff895e4be1a800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1769076247.664007:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.117@tcp: connect to NID 192.168.204.117@tcp last attempt 1930 00000100:00080000:0.0:1769076247.664026:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.117@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1769076247.664048:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.117@tcp: import ffff895e4be1a800 using connection 192.168.204.117@tcp/192.168.204.117@tcp 00000100:00100000:0.0:1769076247.664136:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff895e4383fb80 x1855008961348096/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1769076247.664189:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 73a2201c-08e3-493e-ad2e-98b4627bffd5->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1769076247.664194:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1769076247.664197:0:8:0:(import.c:65:import_set_state_nolock()) ffff895e42e8c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1769076247.664202:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: connect to NID 192.168.204.117@tcp last attempt 1930 00000100:00080000:0.0:1769076247.664207:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1769076247.664213:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: import ffff895e42e8c000 using connection 192.168.204.117@tcp/192.168.204.117@tcp 00000008:00100000:0.0:1769076247.664226:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681347 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1769076247.664242:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff895e4383f100 x1855008961348224/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1769076247.664256:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 73a2201c-08e3-493e-ad2e-98b4627bffd5->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1769076247.664260:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1769076247.664263:0:8:0:(import.c:65:import_set_state_nolock()) ffff895e42e89800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1769076247.664267:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: connect to NID 192.168.204.117@tcp last attempt 1930 00000100:00080000:0.0:1769076247.664270:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1769076247.664274:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: import ffff895e42e89800 using connection 192.168.204.117@tcp/192.168.204.117@tcp 00000008:00100000:0.0:1769076247.664307:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681347 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1769076247.664336:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff895e4383c380 x1855008961348352/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1769076247.664356:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e4383fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:b0abecde-06ba-4498-83fd-7513d38e1d30:2402:1855008961348096:192.168.204.117@tcp:250:kworker.0 00000800:00000100:3.0:1769076247.664493:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp 00000100:00100000:3.0:1769076247.664604:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769076247/real 1769076247] req@ffff895e4383fb80 x1855008961348096/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 1 dl 1769076272 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1769076247.664641:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff895e4383fb80 x1855008961348096/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 1 dl 1769076272 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1769076247.664663:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e4be1a800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1769076247.664672:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.117@tcp failed (-110) 00000100:00100000:3.0:1769076247.664680:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e4383fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:b0abecde-06ba-4498-83fd-7513d38e1d30:2402:1855008961348096:192.168.204.117@tcp:250:kworker.0 00000100:00100000:3.0:1769076247.664697:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e4383f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961348224:192.168.204.117@tcp:38:kworker.0 00000800:00000100:3.0:1769076247.664742:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp 00000100:00100000:3.0:1769076247.665020:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769076247/real 1769076247] req@ffff895e4383f100 x1855008961348224/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769076272 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1769076247.665050:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff895e4383f100 x1855008961348224/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769076272 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1769076247.665086:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e42e8c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1769076247.665126:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.117@tcp failed (-110) 00000100:00100000:3.0:1769076247.665136:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e4383f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961348224:192.168.204.117@tcp:38:kworker.0 00000100:00100000:3.0:1769076247.665187:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e4383c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961348352:192.168.204.117@tcp:38:kworker.0 00000800:00000100:3.0:1769076247.665237:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp 00000100:00100000:3.0:1769076247.665262:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769076247/real 1769076247] req@ffff895e4383c380 x1855008961348352/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769076272 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1769076247.665284:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff895e4383c380 x1855008961348352/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769076272 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1769076247.665300:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e42e89800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1769076247.665307:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.204.117@tcp failed (-110) 00000100:00100000:3.0:1769076247.665314:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e4383c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961348352:192.168.204.117@tcp:38:kworker.0 00010000:00010000:1.0:1769076248.367945:0:6653:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769076249.392091:0:6172:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1769076250.415999:0:6173:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769076251.439975:0:6653:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769076252.464008:0:6172:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1769076252.784354:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) b0abecde-06ba-4498-83fd-7513d38e1d30->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1769076252.784375:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1769076252.784382:0:8:0:(import.c:65:import_set_state_nolock()) ffff895e4be1a800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1769076252.784400:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.117@tcp: connect to NID 192.168.204.117@tcp last attempt 1935 00000100:00080000:3.0:1769076252.784419:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.117@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1769076252.784433:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.117@tcp: import ffff895e4be1a800 using connection 192.168.204.117@tcp/192.168.204.117@tcp 00000100:00100000:3.0:1769076252.784491:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff895e453d5500 x1855008961348480/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1769076252.784551:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 73a2201c-08e3-493e-ad2e-98b4627bffd5->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1769076252.784590:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1769076252.784598:0:8:0:(import.c:65:import_set_state_nolock()) ffff895e42e8c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1769076252.784598:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e453d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:b0abecde-06ba-4498-83fd-7513d38e1d30:2402:1855008961348480:192.168.204.117@tcp:250:kworker.0 00000100:00080000:3.0:1769076252.784614:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: connect to NID 192.168.204.117@tcp last attempt 1935 00000100:00080000:3.0:1769076252.784624:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1769076252.784632:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: import ffff895e42e8c000 using connection 192.168.204.117@tcp/192.168.204.117@tcp 00000008:00100000:3.0:1769076252.784649:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681347 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1769076252.784679:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff895e453d7b80 x1855008961348608/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1769076252.784700:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 73a2201c-08e3-493e-ad2e-98b4627bffd5->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000800:00000100:0.0:1769076252.784710:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp 00000100:00080000:3.0:1769076252.784712:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1769076252.784720:0:8:0:(import.c:65:import_set_state_nolock()) ffff895e42e89800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1769076252.784729:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: connect to NID 192.168.204.117@tcp last attempt 1935 00000100:00080000:3.0:1769076252.784738:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1769076252.784746:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: import ffff895e42e89800 using connection 192.168.204.117@tcp/192.168.204.117@tcp 00000008:00100000:3.0:1769076252.784755:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681347 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1769076252.784821:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769076252/real 1769076252] req@ffff895e453d5500 x1855008961348480/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 1 dl 1769076277 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1769076252.784841:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff895e453d7100 x1855008961348736/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1769076252.784861:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff895e453d5500 x1855008961348480/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 1 dl 1769076277 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1769076252.784881:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e4be1a800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1769076252.784891:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.117@tcp failed (-110) 00000100:00100000:0.0:1769076252.784900:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e453d5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:b0abecde-06ba-4498-83fd-7513d38e1d30:2402:1855008961348480:192.168.204.117@tcp:250:kworker.0 00000100:00100000:0.0:1769076252.784951:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e453d7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961348608:192.168.204.117@tcp:38:kworker.0 00000800:00000100:0.0:1769076252.784996:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp 00000100:00100000:0.0:1769076252.785051:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769076252/real 1769076252] req@ffff895e453d7b80 x1855008961348608/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769076277 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1769076252.785074:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff895e453d7b80 x1855008961348608/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769076277 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1769076252.785089:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e42e8c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1769076252.785094:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.117@tcp failed (-110) 00000100:00100000:0.0:1769076252.785099:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e453d7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961348608:192.168.204.117@tcp:38:kworker.0 00000100:00100000:0.0:1769076252.785125:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e453d7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961348736:192.168.204.117@tcp:38:kworker.0 00000800:00000100:0.0:1769076252.785166:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp 00000100:00100000:0.0:1769076252.785188:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769076252/real 1769076252] req@ffff895e453d7100 x1855008961348736/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769076277 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1769076252.785209:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff895e453d7100 x1855008961348736/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769076277 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1769076252.785221:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e42e89800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1769076252.785228:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.204.117@tcp failed (-110) 00000100:00100000:0.0:1769076252.785236:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e453d7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961348736:192.168.204.117@tcp:38:kworker.0 00010000:00010000:0.0:1769076253.488070:0:6173:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769076254.512017:0:6653:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769076255.536037:0:6172:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1769076256.560041:0:6173:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1769076256.658037:0:27042:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity-sec test_22: @@@@@@ FAIL: adding fops nodemaps failed 1 00010000:00010000:1.0:1769076257.584085:0:6653:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1769076257.903975:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) b0abecde-06ba-4498-83fd-7513d38e1d30->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1769076257.903994:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:2.0:1769076257.904001:0:8:0:(import.c:65:import_set_state_nolock()) ffff895e4be1a800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1769076257.904019:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.117@tcp: connect to NID 192.168.204.117@tcp last attempt 1940 00000100:00080000:2.0:1769076257.904039:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.117@tcp: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1769076257.904054:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.117@tcp: import ffff895e4be1a800 using connection 192.168.204.117@tcp/192.168.204.117@tcp 00000100:00100000:2.0:1769076257.904118:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff895e44e2aa00 x1855008961348864/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1769076257.904159:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 73a2201c-08e3-493e-ad2e-98b4627bffd5->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1769076257.904164:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1769076257.904166:0:8:0:(import.c:65:import_set_state_nolock()) ffff895e42e8c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1769076257.904171:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: connect to NID 192.168.204.117@tcp last attempt 1940 00000100:00080000:2.0:1769076257.904176:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1769076257.904181:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: import ffff895e42e8c000 using connection 192.168.204.117@tcp/192.168.204.117@tcp 00000008:00100000:2.0:1769076257.904195:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681347 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1769076257.904212:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff895e44e2a300 x1855008961348992/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1769076257.904226:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 73a2201c-08e3-493e-ad2e-98b4627bffd5->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1769076257.904231:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:2.0:1769076257.904236:0:8:0:(import.c:65:import_set_state_nolock()) ffff895e42e89800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1769076257.904239:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e44e2aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:b0abecde-06ba-4498-83fd-7513d38e1d30:2402:1855008961348864:192.168.204.117@tcp:250:kworker.0 00000100:00080000:2.0:1769076257.904245:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: connect to NID 192.168.204.117@tcp last attempt 1940 00000100:00080000:2.0:1769076257.904254:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1769076257.904260:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: import ffff895e42e89800 using connection 192.168.204.117@tcp/192.168.204.117@tcp 00000008:00100000:2.0:1769076257.904266:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681347 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1769076257.904319:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff895e44e2b100 x1855008961349120/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000800:00000100:0.0:1769076257.904355:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp 00000100:00100000:0.0:1769076257.904426:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769076257/real 1769076257] req@ffff895e44e2aa00 x1855008961348864/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 1 dl 1769076282 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1769076257.904461:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff895e44e2aa00 x1855008961348864/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 1 dl 1769076282 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1769076257.904489:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e4be1a800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1769076257.904496:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.117@tcp failed (-110) 00000100:00100000:0.0:1769076257.904504:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e44e2aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:b0abecde-06ba-4498-83fd-7513d38e1d30:2402:1855008961348864:192.168.204.117@tcp:250:kworker.0 00000100:00100000:0.0:1769076257.904553:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e44e2a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961348992:192.168.204.117@tcp:38:kworker.0 00000800:00000100:0.0:1769076257.904640:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp 00000100:00100000:0.0:1769076257.904669:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769076257/real 1769076257] req@ffff895e44e2a300 x1855008961348992/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769076282 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1769076257.904699:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff895e44e2a300 x1855008961348992/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769076282 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1769076257.904715:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e42e8c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1769076257.904721:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.117@tcp failed (-110) 00000100:00100000:0.0:1769076257.904727:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e44e2a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961348992:192.168.204.117@tcp:38:kworker.0 00000100:00100000:0.0:1769076257.904740:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e44e2b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961349120:192.168.204.117@tcp:38:kworker.0 00000800:00000100:0.0:1769076257.904839:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp 00000100:00100000:0.0:1769076257.904863:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769076257/real 1769076257] req@ffff895e44e2b100 x1855008961349120/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769076282 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1769076257.904887:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff895e44e2b100 x1855008961349120/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769076282 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:0.0:1769076257.904902:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e42e89800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1769076257.904909:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.204.117@tcp failed (-110) 00000100:00100000:0.0:1769076257.904915:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e44e2b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961349120:192.168.204.117@tcp:38:kworker.0 00010000:00010000:1.0:1769076258.608029:0:6172:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769076259.631960:0:6173:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 197 lines, 197 kept, 0 dropped, 0 bad.