00010000:00010000:2.0F:1769075912.495903:0:6173:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0F:1769075913.519932:0:6653:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769075914.543872:0:6172:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1769075914.864019:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) b0abecde-06ba-4498-83fd-7513d38e1d30->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1769075914.864028:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:1.0:1769075914.864031:0:37:0:(import.c:65:import_set_state_nolock()) ffff895e4be1a800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1769075914.864041:0:37:0:(import.c:509:import_select_connection()) MGC192.168.204.117@tcp: connect to NID 192.168.204.117@tcp last attempt 1597 00000100:00080000:1.0:1769075914.864050:0:37:0:(import.c:588:import_select_connection()) MGC192.168.204.117@tcp: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1769075914.864056:0:37:0:(import.c:623:import_select_connection()) MGC192.168.204.117@tcp: import ffff895e4be1a800 using connection 192.168.204.117@tcp/192.168.204.117@tcp 00000100:00100000:1.0:1769075914.864088:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff895e4713df80 x1855008961323136/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1769075914.864116:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 73a2201c-08e3-493e-ad2e-98b4627bffd5->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1769075914.864120:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1769075914.864122:0:37:0:(import.c:65:import_set_state_nolock()) ffff895e42e8c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1769075914.864127:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: connect to NID 192.168.204.117@tcp last attempt 1597 00000100:00080000:1.0:1769075914.864131:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1769075914.864136:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: import ffff895e42e8c000 using connection 192.168.204.117@tcp/192.168.204.117@tcp 00000008:00100000:1.0:1769075914.864147:0:37:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681347 ocd_grant: 2097152, lost: 0. 00000100:00100000:2.0:1769075914.864160:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e4713df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:b0abecde-06ba-4498-83fd-7513d38e1d30:2402:1855008961323136:192.168.204.117@tcp:250:kworker.0 00000100:00100000:1.0:1769075914.864172:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff895e4713c380 x1855008961323264/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1769075914.864186:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 73a2201c-08e3-493e-ad2e-98b4627bffd5->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1769075914.864189:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:1.0:1769075914.864191:0:37:0:(import.c:65:import_set_state_nolock()) ffff895e42e89800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1769075914.864196:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: connect to NID 192.168.204.117@tcp last attempt 1597 00000100:00080000:1.0:1769075914.864200:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1769075914.864205:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: import ffff895e42e89800 using connection 192.168.204.117@tcp/192.168.204.117@tcp 00000008:00100000:1.0:1769075914.864225:0:37:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681347 ocd_grant: 2097152, lost: 0. 00000800:00000100:2.0:1769075914.864231:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp 00000100:00100000:1.0:1769075914.864241:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff895e4713d180 x1855008961323392/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1769075914.864283:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769075914/real 1769075914] req@ffff895e4713df80 x1855008961323136/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 1 dl 1769075939 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1769075914.864305:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff895e4713df80 x1855008961323136/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 1 dl 1769075939 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1769075914.864316:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e4be1a800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1769075914.864322:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.117@tcp failed (-110) 00000100:00100000:2.0:1769075914.864327:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e4713df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:b0abecde-06ba-4498-83fd-7513d38e1d30:2402:1855008961323136:192.168.204.117@tcp:250:kworker.0 00000100:00100000:2.0:1769075914.864375:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e4713c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961323264:192.168.204.117@tcp:38:kworker.0 00000800:00000100:2.0:1769075914.864407:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp 00000100:00100000:2.0:1769075914.864425:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769075914/real 1769075914] req@ffff895e4713c380 x1855008961323264/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769075939 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1769075914.864439:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff895e4713c380 x1855008961323264/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769075939 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1769075914.864448:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e42e8c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1769075914.864452:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.117@tcp failed (-110) 00000100:00100000:2.0:1769075914.864457:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e4713c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961323264:192.168.204.117@tcp:38:kworker.0 00000100:00100000:2.0:1769075914.864464:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e4713d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961323392:192.168.204.117@tcp:38:kworker.0 00000800:00000100:2.0:1769075914.864489:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp 00000100:00100000:2.0:1769075914.864501:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769075914/real 1769075914] req@ffff895e4713d180 x1855008961323392/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769075939 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1769075914.864530:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff895e4713d180 x1855008961323392/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769075939 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1769075914.864540:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e42e89800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1769075914.864544:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.204.117@tcp failed (-110) 00000100:00100000:2.0:1769075914.864547:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e4713d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961323392:192.168.204.117@tcp:38:kworker.0 00010000:00010000:0.0F:1769075915.567892:0:6173:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769075916.591891:0:6653:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769075917.615889:0:6172:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0F:1769075918.396098:0:18498:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-sec test 10d: verfify nodemap range format '*@' support ========================================================== 04:58:37 (1769075917) 00010000:00010000:1.0:1769075918.639922:0:6173:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769075919.663990:0:6653:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1769075919.983882:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) b0abecde-06ba-4498-83fd-7513d38e1d30->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1769075919.983894:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1769075919.983898:0:37:0:(import.c:65:import_set_state_nolock()) ffff895e4be1a800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1769075919.983909:0:37:0:(import.c:509:import_select_connection()) MGC192.168.204.117@tcp: connect to NID 192.168.204.117@tcp last attempt 1602 00000100:00080000:0.0:1769075919.983922:0:37:0:(import.c:588:import_select_connection()) MGC192.168.204.117@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1769075919.983930:0:37:0:(import.c:623:import_select_connection()) MGC192.168.204.117@tcp: import ffff895e4be1a800 using connection 192.168.204.117@tcp/192.168.204.117@tcp 00000100:00100000:0.0:1769075919.983968:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff895e4383df80 x1855008961323520/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1769075919.983998:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 73a2201c-08e3-493e-ad2e-98b4627bffd5->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1769075919.984001:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1769075919.984002:0:37:0:(import.c:65:import_set_state_nolock()) ffff895e42e8c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1769075919.984005:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: connect to NID 192.168.204.117@tcp last attempt 1602 00000100:00080000:0.0:1769075919.984008:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1769075919.984010:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: import ffff895e42e8c000 using connection 192.168.204.117@tcp/192.168.204.117@tcp 00000008:00100000:0.0:1769075919.984020:0:37:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681347 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1769075919.984029:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff895e4383ed80 x1855008961323648/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1769075919.984037:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 73a2201c-08e3-493e-ad2e-98b4627bffd5->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1769075919.984040:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1769075919.984041:0:37:0:(import.c:65:import_set_state_nolock()) ffff895e42e89800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1769075919.984043:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: connect to NID 192.168.204.117@tcp last attempt 1602 00000100:00080000:0.0:1769075919.984045:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1769075919.984048:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: import ffff895e42e89800 using connection 192.168.204.117@tcp/192.168.204.117@tcp 00000008:00100000:0.0:1769075919.984051:0:37:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681347 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1769075919.984084:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff895e4383d180 x1855008961323776/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1769075919.984104:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e4383df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:b0abecde-06ba-4498-83fd-7513d38e1d30:2402:1855008961323520:192.168.204.117@tcp:250:kworker.0 00000800:00000100:2.0:1769075919.984190:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp 00000100:00100000:2.0:1769075919.984241:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769075919/real 1769075919] req@ffff895e4383df80 x1855008961323520/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 1 dl 1769075944 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1769075919.984261:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff895e4383df80 x1855008961323520/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 1 dl 1769075944 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1769075919.984272:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e4be1a800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1769075919.984277:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.117@tcp failed (-110) 00000100:00100000:2.0:1769075919.984282:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e4383df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:b0abecde-06ba-4498-83fd-7513d38e1d30:2402:1855008961323520:192.168.204.117@tcp:250:kworker.0 00000100:00100000:2.0:1769075919.984290:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e4383ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961323648:192.168.204.117@tcp:38:kworker.0 00000800:00000100:2.0:1769075919.984314:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp 00000100:00100000:2.0:1769075919.984328:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769075919/real 1769075919] req@ffff895e4383ed80 x1855008961323648/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769075944 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1769075919.984342:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff895e4383ed80 x1855008961323648/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769075944 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1769075919.984374:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e42e8c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1769075919.984380:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.117@tcp failed (-110) 00000100:00100000:2.0:1769075919.984386:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e4383ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961323648:192.168.204.117@tcp:38:kworker.0 00000100:00100000:2.0:1769075919.984415:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e4383d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961323776:192.168.204.117@tcp:38:kworker.0 00000800:00000100:2.0:1769075919.984438:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp 00000100:00100000:2.0:1769075919.984453:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769075919/real 1769075919] req@ffff895e4383d180 x1855008961323776/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769075944 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1769075919.984470:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff895e4383d180 x1855008961323776/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769075944 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1769075919.984478:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e42e89800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1769075919.984481:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.204.117@tcp failed (-110) 00000100:00100000:2.0:1769075919.984485:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e4383d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961323776:192.168.204.117@tcp:38:kworker.0 00010000:00010000:1.0:1769075920.687908:0:6172:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769075921.711875:0:6173:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769075922.735942:0:6653:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769075923.759872:0:6172:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769075924.783934:0:6173:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1769075925.103875:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) b0abecde-06ba-4498-83fd-7513d38e1d30->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1769075925.103882:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:0.0:1769075925.103885:0:37:0:(import.c:65:import_set_state_nolock()) ffff895e4be1a800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1769075925.103895:0:37:0:(import.c:509:import_select_connection()) MGC192.168.204.117@tcp: connect to NID 192.168.204.117@tcp last attempt 1607 00000100:00080000:0.0:1769075925.103906:0:37:0:(import.c:588:import_select_connection()) MGC192.168.204.117@tcp: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1769075925.103913:0:37:0:(import.c:623:import_select_connection()) MGC192.168.204.117@tcp: import ffff895e4be1a800 using connection 192.168.204.117@tcp/192.168.204.117@tcp 00000100:00100000:0.0:1769075925.103947:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff895e4383d180 x1855008961323904/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1769075925.103973:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 73a2201c-08e3-493e-ad2e-98b4627bffd5->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1769075925.103978:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1769075925.103989:0:37:0:(import.c:65:import_set_state_nolock()) ffff895e42e8c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1769075925.103995:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: connect to NID 192.168.204.117@tcp last attempt 1607 00000100:00080000:0.0:1769075925.104000:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1769075925.104005:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e4383d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:b0abecde-06ba-4498-83fd-7513d38e1d30:2402:1855008961323904:192.168.204.117@tcp:250:kworker.0 00000100:00080000:0.0:1769075925.104008:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: import ffff895e42e8c000 using connection 192.168.204.117@tcp/192.168.204.117@tcp 00000008:00100000:0.0:1769075925.104023:0:37:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681347 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1769075925.104041:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff895e4383ed80 x1855008961324032/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1769075925.104054:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 73a2201c-08e3-493e-ad2e-98b4627bffd5->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1769075925.104057:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:0.0:1769075925.104059:0:37:0:(import.c:65:import_set_state_nolock()) ffff895e42e89800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1769075925.104063:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: connect to NID 192.168.204.117@tcp last attempt 1607 00000100:00080000:0.0:1769075925.104068:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: tried all connections, increasing latency to 20s 00000800:00000100:3.0:1769075925.104082:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp 00000100:00080000:0.0:1769075925.104091:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: import ffff895e42e89800 using connection 192.168.204.117@tcp/192.168.204.117@tcp 00000008:00100000:0.0:1769075925.104102:0:37:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681347 ocd_grant: 2097152, lost: 0. 00000100:00100000:0.0:1769075925.104121:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff895e4383df80 x1855008961324160/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1769075925.104149:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769075925/real 1769075925] req@ffff895e4383d180 x1855008961323904/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 1 dl 1769075950 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1769075925.104169:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff895e4383d180 x1855008961323904/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 1 dl 1769075950 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1769075925.104185:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e4be1a800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1769075925.104189:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.117@tcp failed (-110) 00000100:00100000:3.0:1769075925.104193:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e4383d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:b0abecde-06ba-4498-83fd-7513d38e1d30:2402:1855008961323904:192.168.204.117@tcp:250:kworker.0 00000100:00100000:3.0:1769075925.104220:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e4383ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961324032:192.168.204.117@tcp:38:kworker.0 00000800:00000100:3.0:1769075925.104245:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp 00000100:00100000:3.0:1769075925.104259:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769075925/real 1769075925] req@ffff895e4383ed80 x1855008961324032/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769075950 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1769075925.104270:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff895e4383ed80 x1855008961324032/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769075950 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1769075925.104279:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e42e8c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1769075925.104283:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.117@tcp failed (-110) 00000100:00100000:3.0:1769075925.104306:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e4383ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961324032:192.168.204.117@tcp:38:kworker.0 00000100:00100000:3.0:1769075925.104313:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e4383df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961324160:192.168.204.117@tcp:38:kworker.0 00000800:00000100:3.0:1769075925.104336:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp 00000100:00100000:3.0:1769075925.104347:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769075925/real 1769075925] req@ffff895e4383df80 x1855008961324160/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769075950 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:3.0:1769075925.104369:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff895e4383df80 x1855008961324160/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769075950 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:3.0:1769075925.104377:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e42e89800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1769075925.104380:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.204.117@tcp failed (-110) 00000100:00100000:3.0:1769075925.104384:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e4383df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961324160:192.168.204.117@tcp:38:kworker.0 00010000:00010000:1.0:1769075925.807894:0:6653:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769075926.831892:0:6172:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1769075927.584349:0:18658:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity-sec test_10d: @@@@@@ FAIL: Add nodemap_lu8913 failed 00010000:00010000:1.0:1769075927.855876:0:6173:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769075928.879911:0:6653:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1769075929.903870:0:6172:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1769075930.223963:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) b0abecde-06ba-4498-83fd-7513d38e1d30->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1769075930.223972:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery 00000100:00080000:3.0:1769075930.223975:0:8:0:(import.c:65:import_set_state_nolock()) ffff895e4be1a800 MGS: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1769075930.223983:0:8:0:(import.c:509:import_select_connection()) MGC192.168.204.117@tcp: connect to NID 192.168.204.117@tcp last attempt 1612 00000100:00080000:3.0:1769075930.223992:0:8:0:(import.c:588:import_select_connection()) MGC192.168.204.117@tcp: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1769075930.223998:0:8:0:(import.c:623:import_select_connection()) MGC192.168.204.117@tcp: import ffff895e4be1a800 using connection 192.168.204.117@tcp/192.168.204.117@tcp 00000100:00100000:3.0:1769075930.224026:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff895e453d6680 x1855008961324288/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1769075930.224053:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 73a2201c-08e3-493e-ad2e-98b4627bffd5->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1769075930.224055:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1769075930.224057:0:8:0:(import.c:65:import_set_state_nolock()) ffff895e42e8c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1769075930.224060:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: connect to NID 192.168.204.117@tcp last attempt 1612 00000100:00080000:3.0:1769075930.224064:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1769075930.224067:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: import ffff895e42e8c000 using connection 192.168.204.117@tcp/192.168.204.117@tcp 00000008:00100000:3.0:1769075930.224074:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681347 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1769075930.224084:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff895e453d6300 x1855008961324416/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1769075930.224108:0:8:0:(pinger.c:229:ptlrpc_pinger_process_import()) 73a2201c-08e3-493e-ad2e-98b4627bffd5->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1769075930.224110:0:8:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery 00000100:00080000:3.0:1769075930.224112:0:8:0:(import.c:65:import_set_state_nolock()) ffff895e42e89800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1769075930.224115:0:8:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: connect to NID 192.168.204.117@tcp last attempt 1612 00000100:00080000:3.0:1769075930.224119:0:8:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1769075930.224122:0:8:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: import ffff895e42e89800 using connection 192.168.204.117@tcp/192.168.204.117@tcp 00000008:00100000:3.0:1769075930.224126:0:8:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681347 ocd_grant: 2097152, lost: 0. 00000100:00100000:3.0:1769075930.224137:0:8:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff895e453d4380 x1855008961324544/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1769075930.224137:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e453d6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:b0abecde-06ba-4498-83fd-7513d38e1d30:2402:1855008961324288:192.168.204.117@tcp:250:kworker.0 00000800:00000100:2.0:1769075930.224206:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp 00000100:00100000:2.0:1769075930.224246:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769075930/real 1769075930] req@ffff895e453d6680 x1855008961324288/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 1 dl 1769075955 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1769075930.224268:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff895e453d6680 x1855008961324288/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 1 dl 1769075955 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1769075930.224279:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e4be1a800 MGS: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1769075930.224292:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.117@tcp failed (-110) 00000100:00100000:2.0:1769075930.224296:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e453d6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:b0abecde-06ba-4498-83fd-7513d38e1d30:2402:1855008961324288:192.168.204.117@tcp:250:kworker.0 00000100:00100000:2.0:1769075930.224306:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e453d6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961324416:192.168.204.117@tcp:38:kworker.0 00000800:00000100:2.0:1769075930.224340:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp 00000100:00100000:2.0:1769075930.224357:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769075930/real 1769075930] req@ffff895e453d6300 x1855008961324416/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769075955 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1769075930.224373:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff895e453d6300 x1855008961324416/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769075955 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1769075930.224383:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e42e8c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1769075930.224386:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.117@tcp failed (-110) 00000100:00100000:2.0:1769075930.224390:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e453d6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961324416:192.168.204.117@tcp:38:kworker.0 00000100:00100000:2.0:1769075930.224416:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e453d4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961324544:192.168.204.117@tcp:38:kworker.0 00000800:00000100:2.0:1769075930.224458:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp 00000100:00100000:2.0:1769075930.224471:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769075930/real 1769075930] req@ffff895e453d4380 x1855008961324544/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769075955 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1769075930.224485:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING) req@ffff895e453d4380 x1855008961324544/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769075955 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 00000100:00080000:2.0:1769075930.224494:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e42e89800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1769075930.224497:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.204.117@tcp failed (-110) 00000100:00100000:2.0:1769075930.224501:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e453d4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961324544:192.168.204.117@tcp:38:kworker.0 Debug log: 196 lines, 196 kept, 0 dropped, 0 bad.