00010000:00010000:1.0F:1769076167.471926:0:6173:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0F:1769076168.496028:0:6653:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1769076169.520023:0:6172:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:0.0F:1769076170.543958:0:6173:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1769076170.863897:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) b0abecde-06ba-4498-83fd-7513d38e1d30->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1769076170.863910:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1769076170.863916:0:38:0:(import.c:65:import_set_state_nolock()) ffff895e4be1a800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1769076170.863928:0:38:0:(import.c:509:import_select_connection()) MGC192.168.204.117@tcp: connect to NID 192.168.204.117@tcp last attempt 1853
00000100:00080000:2.0:1769076170.863941:0:38:0:(import.c:588:import_select_connection()) MGC192.168.204.117@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1769076170.863950:0:38:0:(import.c:623:import_select_connection()) MGC192.168.204.117@tcp: import ffff895e4be1a800 using connection 192.168.204.117@tcp/192.168.204.117@tcp
00000100:00100000:2.0:1769076170.863991:0:38:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff895e44e2bb80 x1855008961342336/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1769076170.864033:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 73a2201c-08e3-493e-ad2e-98b4627bffd5->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1769076170.864038:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1769076170.864040:0:38:0:(import.c:65:import_set_state_nolock()) ffff895e42e8c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1769076170.864045:0:38:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: connect to NID 192.168.204.117@tcp last attempt 1853
00000100:00080000:2.0:1769076170.864049:0:38:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1769076170.864053:0:38:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: import ffff895e42e8c000 using connection 192.168.204.117@tcp/192.168.204.117@tcp
00000008:00100000:2.0:1769076170.864066:0:38:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681347 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1769076170.864089:0:38:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff895e44e2b800 x1855008961342464/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1769076170.864108:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 73a2201c-08e3-493e-ad2e-98b4627bffd5->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1769076170.864114:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1769076170.864118:0:38:0:(import.c:65:import_set_state_nolock()) ffff895e42e89800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1769076170.864126:0:38:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: connect to NID 192.168.204.117@tcp last attempt 1853
00000100:00080000:2.0:1769076170.864133:0:38:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1769076170.864142:0:38:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: import ffff895e42e89800 using connection 192.168.204.117@tcp/192.168.204.117@tcp
00000008:00100000:2.0:1769076170.864175:0:38:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681347 ocd_grant: 2097152, lost: 0.
00000100:00100000:1.0:1769076170.864192:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e44e2bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:b0abecde-06ba-4498-83fd-7513d38e1d30:2402:1855008961342336:192.168.204.117@tcp:250:kworker.0
00000100:00100000:2.0:1769076170.864200:0:38:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff895e44e29c00 x1855008961342592/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1769076170.864296:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp
00000100:00100000:1.0:1769076170.864354:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769076170/real 1769076170]  req@ffff895e44e2bb80 x1855008961342336/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 1 dl 1769076195 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1769076170.864424:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff895e44e2bb80 x1855008961342336/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 1 dl 1769076195 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1769076170.864442:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e4be1a800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1769076170.864449:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.117@tcp failed (-110)
00000100:00100000:1.0:1769076170.864457:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e44e2bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:b0abecde-06ba-4498-83fd-7513d38e1d30:2402:1855008961342336:192.168.204.117@tcp:250:kworker.0
00000100:00100000:1.0:1769076170.864472:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e44e2b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961342464:192.168.204.117@tcp:38:kworker.0
00000800:00000100:1.0:1769076170.864516:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp
00000100:00100000:1.0:1769076170.864541:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769076170/real 1769076170]  req@ffff895e44e2b800 x1855008961342464/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769076195 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1769076170.864566:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff895e44e2b800 x1855008961342464/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769076195 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1769076170.864580:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e42e8c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1769076170.864587:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.117@tcp failed (-110)
00000100:00100000:1.0:1769076170.864594:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e44e2b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961342464:192.168.204.117@tcp:38:kworker.0
00000100:00100000:1.0:1769076170.864649:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e44e29c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961342592:192.168.204.117@tcp:38:kworker.0
00000800:00000100:1.0:1769076170.864691:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp
00000100:00100000:1.0:1769076170.864713:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769076170/real 1769076170]  req@ffff895e44e29c00 x1855008961342592/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769076195 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1769076170.864822:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff895e44e29c00 x1855008961342592/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769076195 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1769076170.864837:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e42e89800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1769076170.864844:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.204.117@tcp failed (-110)
00000100:00100000:1.0:1769076170.864850:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e44e29c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961342592:192.168.204.117@tcp:38:kworker.0
00010000:00010000:2.0:1769076171.567886:0:6653:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1769076172.592016:0:6172:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1769076173.616935:0:6173:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000001:02000400:3.0F:1769076173.649218:0:24217:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-sec test 18: test nodemap mapped_noadmin fileops ========================================================== 05:02:53 (1769076173)
00010000:00010000:2.0:1769076174.639898:0:6653:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1769076175.663990:0:6172:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:2.0:1769076175.983844:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) b0abecde-06ba-4498-83fd-7513d38e1d30->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1769076175.983858:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:2.0:1769076175.983863:0:37:0:(import.c:65:import_set_state_nolock()) ffff895e4be1a800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1769076175.983876:0:37:0:(import.c:509:import_select_connection()) MGC192.168.204.117@tcp: connect to NID 192.168.204.117@tcp last attempt 1858
00000100:00080000:2.0:1769076175.983891:0:37:0:(import.c:588:import_select_connection()) MGC192.168.204.117@tcp: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1769076175.983901:0:37:0:(import.c:623:import_select_connection()) MGC192.168.204.117@tcp: import ffff895e4be1a800 using connection 192.168.204.117@tcp/192.168.204.117@tcp
00000100:00100000:2.0:1769076175.983964:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff895e44e28a80 x1855008961342720/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1769076175.984002:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 73a2201c-08e3-493e-ad2e-98b4627bffd5->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1769076175.984007:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:2.0:1769076175.984009:0:37:0:(import.c:65:import_set_state_nolock()) ffff895e42e8c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1769076175.984016:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: connect to NID 192.168.204.117@tcp last attempt 1858
00000100:00080000:2.0:1769076175.984023:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1769076175.984029:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: import ffff895e42e8c000 using connection 192.168.204.117@tcp/192.168.204.117@tcp
00000008:00100000:2.0:1769076175.984042:0:37:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681347 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1769076175.984058:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff895e44e29500 x1855008961342848/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:2.0:1769076175.984072:0:37:0:(pinger.c:229:ptlrpc_pinger_process_import()) 73a2201c-08e3-493e-ad2e-98b4627bffd5->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:2.0:1769076175.984096:0:37:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:2.0:1769076175.984098:0:37:0:(import.c:65:import_set_state_nolock()) ffff895e42e89800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:2.0:1769076175.984104:0:37:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: connect to NID 192.168.204.117@tcp last attempt 1858
00000100:00080000:2.0:1769076175.984113:0:37:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: tried all connections, increasing latency to 20s
00000100:00080000:2.0:1769076175.984119:0:37:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: import ffff895e42e89800 using connection 192.168.204.117@tcp/192.168.204.117@tcp
00000008:00100000:2.0:1769076175.984128:0:37:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681347 ocd_grant: 2097152, lost: 0.
00000100:00100000:2.0:1769076175.984145:0:37:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff895e44e29180 x1855008961342976/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1769076175.984172:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e44e28a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:b0abecde-06ba-4498-83fd-7513d38e1d30:2402:1855008961342720:192.168.204.117@tcp:250:kworker.0
00000800:00000100:1.0:1769076175.984338:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp
00000100:00100000:1.0:1769076175.984446:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769076175/real 1769076175]  req@ffff895e44e28a80 x1855008961342720/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 1 dl 1769076200 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1769076175.984497:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff895e44e28a80 x1855008961342720/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 1 dl 1769076200 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1769076175.984514:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e4be1a800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1769076175.984521:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.117@tcp failed (-110)
00000100:00100000:1.0:1769076175.984533:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e44e28a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:b0abecde-06ba-4498-83fd-7513d38e1d30:2402:1855008961342720:192.168.204.117@tcp:250:kworker.0
00000100:00100000:1.0:1769076175.984548:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e44e29500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961342848:192.168.204.117@tcp:38:kworker.0
00000800:00000100:1.0:1769076175.984591:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp
00000100:00100000:1.0:1769076175.984610:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769076175/real 1769076175]  req@ffff895e44e29500 x1855008961342848/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769076200 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1769076175.984629:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff895e44e29500 x1855008961342848/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769076200 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1769076175.984672:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e42e8c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1769076175.984680:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.117@tcp failed (-110)
00000100:00100000:1.0:1769076175.984685:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e44e29500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961342848:192.168.204.117@tcp:38:kworker.0
00000100:00100000:1.0:1769076175.984739:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e44e29180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961342976:192.168.204.117@tcp:38:kworker.0
00000800:00000100:1.0:1769076175.984906:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp
00000100:00100000:1.0:1769076175.984946:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769076175/real 1769076175]  req@ffff895e44e29180 x1855008961342976/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769076200 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1769076175.984973:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff895e44e29180 x1855008961342976/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769076200 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1769076175.984996:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e42e89800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1769076175.985007:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.204.117@tcp failed (-110)
00000100:00100000:1.0:1769076175.985014:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e44e29180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961342976:192.168.204.117@tcp:38:kworker.0
00010000:00010000:1.0:1769076176.687992:0:6173:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1769076177.712022:0:6653:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1769076178.735923:0:6172:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1769076179.759993:0:6173:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1769076180.783936:0:6653:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000100:00080000:0.0:1769076181.103856:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) b0abecde-06ba-4498-83fd-7513d38e1d30->MGS: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1769076181.103865:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) MGS: starting recovery
00000100:00080000:0.0:1769076181.103869:0:38:0:(import.c:65:import_set_state_nolock()) ffff895e4be1a800 MGS: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1769076181.103878:0:38:0:(import.c:509:import_select_connection()) MGC192.168.204.117@tcp: connect to NID 192.168.204.117@tcp last attempt 1863
00000100:00080000:0.0:1769076181.103889:0:38:0:(import.c:588:import_select_connection()) MGC192.168.204.117@tcp: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1769076181.103953:0:38:0:(import.c:623:import_select_connection()) MGC192.168.204.117@tcp: import ffff895e4be1a800 using connection 192.168.204.117@tcp/192.168.204.117@tcp
00000100:00100000:0.0:1769076181.103984:0:38:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff895e4383e680 x1855008961343104/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00080000:0.0:1769076181.104008:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 73a2201c-08e3-493e-ad2e-98b4627bffd5->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1769076181.104011:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery
00000100:00080000:0.0:1769076181.104012:0:38:0:(import.c:65:import_set_state_nolock()) ffff895e42e8c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1769076181.104015:0:38:0:(import.c:509:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: connect to NID 192.168.204.117@tcp last attempt 1863
00000100:00080000:0.0:1769076181.104018:0:38:0:(import.c:588:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1769076181.104021:0:38:0:(import.c:623:import_select_connection()) lustre-MDT0000-mdc-ffff895e42f9a800: import ffff895e42e8c000 using connection 192.168.204.117@tcp/192.168.204.117@tcp
00000008:00100000:0.0:1769076181.104029:0:38:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681347 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1769076181.104039:0:38:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff895e4383d880 x1855008961343232/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000100:00100000:1.0:1769076181.104049:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e4383e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:b0abecde-06ba-4498-83fd-7513d38e1d30:2402:1855008961343104:192.168.204.117@tcp:250:kworker.0
00000100:00080000:0.0:1769076181.104068:0:38:0:(pinger.c:229:ptlrpc_pinger_process_import()) 73a2201c-08e3-493e-ad2e-98b4627bffd5->lustre-MDT0001_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0
00000100:00080000:0.0:1769076181.104072:0:38:0:(pinger.c:237:ptlrpc_pinger_process_import()) lustre-MDT0001_UUID: starting recovery
00000100:00080000:0.0:1769076181.104073:0:38:0:(import.c:65:import_set_state_nolock()) ffff895e42e89800 lustre-MDT0001_UUID: changing import state from DISCONN to CONNECTING
00000100:00080000:0.0:1769076181.104077:0:38:0:(import.c:509:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: connect to NID 192.168.204.117@tcp last attempt 1863
00000100:00080000:0.0:1769076181.104080:0:38:0:(import.c:588:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: tried all connections, increasing latency to 20s
00000100:00080000:0.0:1769076181.104083:0:38:0:(import.c:623:import_select_connection()) lustre-MDT0001-mdc-ffff895e42f9a800: import ffff895e42e89800 using connection 192.168.204.117@tcp/192.168.204.117@tcp
00000008:00100000:0.0:1769076181.104088:0:38:0:(osc_request.c:3784:osc_reconnect()) ocd_connect_flags: 0xae7e5e7be374d3b8 ocd_version: 34681347 ocd_grant: 2097152, lost: 0.
00000100:00100000:0.0:1769076181.104111:0:38:0:(import.c:825:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5)  req@ffff895e4383c700 x1855008961343360/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295
00000800:00000100:1.0:1769076181.104137:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp
00000100:00100000:1.0:1769076181.104170:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769076181/real 1769076181]  req@ffff895e4383e680 x1855008961343104/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 1 dl 1769076206 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1769076181.104185:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff895e4383e680 x1855008961343104/t0(0) o250->MGC192.168.204.117@tcp@192.168.204.117@tcp:26/25 lens 520/544 e 0 to 1 dl 1769076206 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1769076181.104193:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e4be1a800 MGS: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1769076181.104203:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of MGS on 192.168.204.117@tcp failed (-110)
00000100:00100000:1.0:1769076181.104237:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e4383e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:b0abecde-06ba-4498-83fd-7513d38e1d30:2402:1855008961343104:192.168.204.117@tcp:250:kworker.0
00000100:00100000:1.0:1769076181.104261:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e4383d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961343232:192.168.204.117@tcp:38:kworker.0
00000800:00000100:1.0:1769076181.104283:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp
00000100:00100000:1.0:1769076181.104293:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769076181/real 1769076181]  req@ffff895e4383d880 x1855008961343232/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769076206 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1769076181.104302:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff895e4383d880 x1855008961343232/t0(0) o38->lustre-MDT0000-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769076206 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1769076181.104308:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e42e8c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1769076181.104311:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.117@tcp failed (-110)
00000100:00100000:1.0:1769076181.104313:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e4383d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961343232:192.168.204.117@tcp:38:kworker.0
00000100:00100000:1.0:1769076181.104318:0:2402:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff895e4383c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961343360:192.168.204.117@tcp:38:kworker.0
00000800:00000100:1.0:1769076181.104330:0:2402:0:(socklnd_cb.c:976:ksocknal_launch_packet()) No usable routes to 12345-192.168.204.117@tcp
00000100:00100000:1.0:1769076181.104336:0:2402:0:(client.c:2478:ptlrpc_expire_one_request()) @@@ Request sent has failed due to network error: [sent 1769076181/real 1769076181]  req@ffff895e4383c700 x1855008961343360/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769076206 ref 1 fl Rpc:eXNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00100000:1.0:1769076181.104344:0:2402:0:(client.c:2509:ptlrpc_expire_one_request()) @@@ err -110, sent_state=CONNECTING (now=CONNECTING)  req@ffff895e4383c700 x1855008961343360/t0(0) o38->lustre-MDT0001-mdc-ffff895e42f9a800@192.168.204.117@tcp:12/10 lens 520/544 e 0 to 1 dl 1769076206 ref 1 fl Rpc:eXNQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295
00000100:00080000:1.0:1769076181.104349:0:2402:0:(import.c:65:import_set_state_nolock()) ffff895e42e89800 lustre-MDT0001_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:1.0:1769076181.104351:0:2402:0:(import.c:1499:ptlrpc_connect_interpret()) recovery of lustre-MDT0001_UUID on 192.168.204.117@tcp failed (-110)
00000100:00100000:1.0:1769076181.104353:0:2402:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff895e4383c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:73a2201c-08e3-493e-ad2e-98b4627bffd5:2402:1855008961343360:192.168.204.117@tcp:38:kworker.0
00010000:00010000:2.0:1769076181.808056:0:6172:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00000001:02000400:0.0:1769076182.758037:0:24423:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity-sec test_18: @@@@@@ FAIL: adding fops nodemaps failed 1
00010000:00010000:0.0:1769076182.831871:0:6173:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:1.0:1769076183.855970:0:6653:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
00010000:00010000:2.0:1769076184.879919:0:6172:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue)
Debug log: 152 lines, 152 kept, 0 dropped, 0 bad.