00010000:00010000:2.0F:1782488117.892443:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488118.925144:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488119.937757:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0F:1782488120.133440:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4525ad500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3314:1869066300118912:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1782488120.133521:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488120.133628:0:198596:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300118912 00000100:00100000:3.0:1782488120.133635:0:198596:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300118912 00000100:00100000:3.0:1782488120.133689:0:198596:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 425 00000100:00100000:3.0:1782488120.133699:0:198596:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4525aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1869066300118912:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1782488120.133742:0:198596:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 38731776 pending 0 free 3825205248 avail 3823108096 00000100:00100000:3.0:1782488120.133787:0:198596:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4525aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1869066300118912:12345-0@lo:13:osp-pre-0-0.0 Request processed in 89us (265us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488120.133794:0:198596:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 425 00000100:00080000:3.0:1782488120.133812:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1782488120.133880:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4525ac380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300119168:0@lo:400:kworker.0 00000100:00100000:3.0:1782488120.133908:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488120.133927:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb574913800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3316:1869066300119680:0@lo:400:kworker.0 00000100:00100000:3.0:1782488120.133947:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488120.133963:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1782488120.133968:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb574913100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300119296:0@lo:400:kworker.0 00000100:00100000:3.0:1782488120.133986:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488120.134009:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300119168 00000100:00100000:3.0:1782488120.134012:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300119168 00000100:00100000:3.0:1782488120.134034:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 551 00000100:00100000:3.0:1782488120.134039:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579adc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+19:3316:x1869066300119168:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782488120.134069:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579adc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+19:3316:x1869066300119168:12345-0@lo:400:kworker.0 Request processed in 29us (159us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488120.134074:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 551 00000100:00100000:3.0:1782488120.134098:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300119680 00000100:00100000:3.0:1782488120.134101:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300119680 00000100:00100000:3.0:1782488120.134137:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 894 00000100:00100000:3.0:1782488120.134144:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb574911500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3316:x1869066300119680:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782488120.134174:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb574911500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3316:x1869066300119680:12345-0@lo:400:kworker.0 Request processed in 30us (225us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488120.134180:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 894 00000100:00100000:3.0:1782488120.134197:0:213224:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300119296 00000100:00100000:3.0:1782488120.134200:0:213224:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300119296 00000100:00100000:3.0:1782488120.134226:0:213224:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 693 00000100:00100000:3.0:1782488120.134232:0:213224:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb574911880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300119296:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782488120.134261:0:213224:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb574911880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300119296:12345-0@lo:400:kworker.0 Request processed in 29us (273us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488120.134267:0:213224:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 693 00000100:00100000:3.0:1782488120.134289:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb488519f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3315:1869066300119040:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1782488120.134316:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488120.134340:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb574910380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3315:1869066300119552:0@lo:400:kworker.0 00000100:00100000:3.0:1782488120.134358:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488120.134375:0:3315:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1782488120.134379:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb574912a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3315:1869066300119424:0@lo:400:kworker.0 00000100:00100000:3.0:1782488120.134397:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488120.134418:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300119552 00000100:00100000:3.0:1782488120.134421:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300119552 00000100:00100000:3.0:1782488120.134441:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300119424 00000100:00100000:3.0:1782488120.134443:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300119424 00000100:00100000:3.0:1782488120.134458:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 895 00000100:00100000:3.0:1782488120.134463:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3315:x1869066300119552:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782488120.134494:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3315:x1869066300119552:12345-0@lo:400:kworker.0 Request processed in 31us (135us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488120.134509:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 895 00000100:00100000:3.0:1782488120.134519:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 896 00000100:00100000:3.0:1782488120.134524:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3315:x1869066300119424:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782488120.134548:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3315:x1869066300119424:12345-0@lo:400:kworker.0 Request processed in 24us (150us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488120.134553:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 896 00000100:00080000:3.0:1782488120.134594:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0F:1782488120.134596:0:190933:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300119040 00000100:00080000:3.0:1782488120.134598:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1782488120.134605:0:190933:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300119040 00000100:00100000:3.0:1782488120.134635:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb574913100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300119296:0@lo:400:kworker.0 00000100:00100000:3.0:1782488120.134645:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4525ac380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300119168:0@lo:400:kworker.0 00000100:00100000:1.0:1782488120.134650:0:190933:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 426 00000100:00100000:3.0:1782488120.134654:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb574913800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3316:1869066300119680:0@lo:400:kworker.0 00000100:00100000:1.0:1782488120.134655:0:190933:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3315:x1869066300119040:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1782488120.134674:0:190933:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 30343168 pending 0 free 3806330880 avail 3804233728 00000100:00100000:3.0:1782488120.134687:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb574912a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3315:1869066300119424:0@lo:400:kworker.0 00000100:00100000:3.0:1782488120.134696:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb574910380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3315:1869066300119552:0@lo:400:kworker.0 00000100:00100000:1.0:1782488120.134704:0:190933:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3315:x1869066300119040:12345-0@lo:13:osp-pre-1-0.0 Request processed in 49us (387us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488120.134709:0:190933:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 426 00000100:00100000:3.0:1782488120.134711:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb488519f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3315:1869066300119040:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1782488120.134734:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4525ad500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3314:1869066300118912:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1782488120.966277:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1782488121.940936:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:1.0:1782488121.940981:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:1.0:1782488121.941030:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:1.0:1782488121.941067:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:1.0:1782488121.941106:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298639616 00000100:00100000:1.0:1782488121.941116:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298639616 00000100:00100000:1.0:1782488121.941191:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 897 00000100:00100000:1.0:1782488121.941204:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb488519180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2362:x1869066298639616:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:1.0:1782488121.941300:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb488519180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2362:x1869066298639616:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 99us (374us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488121.941309:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 897 00000100:00100000:1.0:1782488121.941344:0:213210:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298639872 00000100:00100000:1.0:1782488121.941348:0:213210:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298639872 00000100:00100000:1.0:1782488121.941384:0:213210:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298639744 00000100:00100000:1.0:1782488121.941388:0:213210:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298639744 00000100:00100000:1.0:1782488121.941421:0:213210:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 694 00000100:00100000:1.0:1782488121.941429:0:213210:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb48851a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2362:x1869066298639872:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:1.0:1782488121.941461:0:213210:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb48851a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2362:x1869066298639872:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 32us (479us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488121.941470:0:213210:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 694 00000100:00100000:1.0:1782488121.941502:0:213210:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 695 00000100:00100000:1.0:1782488121.941511:0:213210:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb488519c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2362:x1869066298639744:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:1.0:1782488121.941540:0:213210:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb488519c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2362:x1869066298639744:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 31us (474us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488121.941577:0:213210:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 695 00000100:00100000:1.0:1782488121.941601:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298639488 00000100:00100000:1.0:1782488121.941604:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298639488 00000100:00100000:1.0:1782488121.941660:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 552 00000100:00100000:1.0:1782488121.941668:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579adf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2362:x1869066298639488:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:1.0:1782488121.941696:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579adf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2362:x1869066298639488:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 29us (666us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488121.941703:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 552 00000100:00080000:1.0:1782488121.941741:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1782488121.941745:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1782488121.941749:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1782488121.986536:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488123.010794:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488124.033730:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488125.058713:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1782488125.249905:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4525ad500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3317:1869066300119808:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1782488125.249975:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488125.250044:0:190932:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300119808 00000100:00100000:3.0:1782488125.250051:0:190932:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300119808 00000100:00100000:3.0:1782488125.250103:0:190932:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 427 00000100:00100000:3.0:1782488125.250112:0:190932:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4525ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1869066300119808:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1782488125.251013:0:190932:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 38731776 pending 0 free 3825205248 avail 3823108096 00000100:00100000:3.0:1782488125.251075:0:190932:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4525ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1869066300119808:12345-0@lo:13:osp-pre-0-0.0 Request processed in 963us (1099us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488125.251084:0:190932:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 427 00000100:00080000:3.0:1782488125.251181:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1782488125.251292:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4525ad500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3317:1869066300119808:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1782488125.251994:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb449fb6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300120064:0@lo:400:kworker.0 00000100:00100000:1.0:1782488125.252034:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488125.252061:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb573552680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3315:1869066300120576:0@lo:400:kworker.0 00000100:00100000:1.0:1782488125.252082:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488125.252099:0:3315:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1782488125.252104:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb488519f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3315:1869066300119936:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1782488125.252123:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488125.252138:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb573552a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3315:1869066300120448:0@lo:400:kworker.0 00000100:00100000:1.0:1782488125.252157:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488125.252196:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300120064 00000100:00100000:1.0:1782488125.252210:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300120064 00000100:00100000:1.0:1782488125.252246:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 553 00000100:00100000:1.0:1782488125.252253:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579adf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+19:3315:x1869066300120064:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488125.252292:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579adf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+19:3315:x1869066300120064:12345-0@lo:400:kworker.0 Request processed in 39us (258us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488125.252299:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 553 00000100:00100000:1.0:1782488125.252331:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300120576 00000100:00100000:1.0:1782488125.252334:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300120576 00000100:00100000:1.0:1782488125.252362:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300120448 00000100:00100000:1.0:1782488125.252365:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300120448 00000100:00100000:1.0:1782488125.252383:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 898 00000100:00100000:1.0:1782488125.252390:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb573553b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3315:x1869066300120576:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488125.252422:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb573553b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3315:x1869066300120576:12345-0@lo:400:kworker.0 Request processed in 33us (339us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488125.252428:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 898 00000100:00100000:1.0:1782488125.252440:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 899 00000100:00100000:1.0:1782488125.252445:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb573550380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3315:x1869066300120448:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488125.252469:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb573550380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3315:x1869066300120448:12345-0@lo:400:kworker.0 Request processed in 24us (311us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488125.252475:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 899 00000100:00100000:1.0:1782488125.252505:0:198596:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300119936 00000100:00100000:1.0:1782488125.252509:0:198596:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300119936 00000100:00100000:1.0:1782488125.252546:0:198596:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 428 00000100:00100000:1.0:1782488125.252552:0:198596:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb573550a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3315:x1869066300119936:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1782488125.252573:0:198596:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 30343168 pending 0 free 3806330880 avail 3804233728 00000100:00100000:1.0:1782488125.252602:0:198596:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb573550a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3315:x1869066300119936:12345-0@lo:13:osp-pre-1-0.0 Request processed in 50us (477us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488125.252645:0:198596:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 428 00000100:00100000:1.0:1782488125.252686:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb488519f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3315:1869066300119936:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1782488125.252696:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb573552a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3315:1869066300120448:0@lo:400:kworker.0 00000100:00100000:1.0:1782488125.252704:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb449fb6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300120064:0@lo:400:kworker.0 00000100:00100000:1.0:1782488125.252712:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb573552680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3315:1869066300120576:0@lo:400:kworker.0 00000100:00100000:1.0:1782488125.252748:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb449fb5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300120192:0@lo:400:kworker.0 00000100:00100000:1.0:1782488125.252773:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488125.252792:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1782488125.252798:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb449fb6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3316:1869066300120320:0@lo:400:kworker.0 00000100:00100000:1.0:1782488125.252818:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488125.252842:0:213211:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300120192 00000100:00100000:1.0:1782488125.252845:0:213211:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300120192 00000100:00100000:1.0:1782488125.252869:0:213211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 696 00000100:00100000:1.0:1782488125.252875:0:213211:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb573552680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300120192:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488125.252906:0:213211:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb573552680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300120192:12345-0@lo:400:kworker.0 Request processed in 32us (132us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488125.252912:0:213211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 696 00000100:00100000:1.0:1782488125.252925:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300120320 00000100:00100000:1.0:1782488125.252928:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300120320 00000100:00100000:1.0:1782488125.252947:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 900 00000100:00100000:1.0:1782488125.252952:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb573552a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1869066300120320:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488125.252976:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb573552a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1869066300120320:12345-0@lo:400:kworker.0 Request processed in 24us (158us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488125.252981:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 900 00000100:00080000:1.0:1782488125.253010:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1782488125.253013:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1782488125.253029:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb449fb6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3316:1869066300120320:0@lo:400:kworker.0 00000100:00100000:1.0:1782488125.253038:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb449fb5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300120192:0@lo:400:kworker.0 00010000:00010000:2.0:1782488126.081743:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1782488127.064465:0:3308:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:3.0:1782488127.064528:0:3308:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:3.0:1782488127.064554:0:3308:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:3.0:1782488127.064583:0:3308:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:3.0:1782488127.064625:0:213384:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298640384 00000100:00100000:3.0:1782488127.064635:0:213384:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298640384 00000100:00100000:3.0:1782488127.064686:0:213384:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298640256 00000100:00100000:3.0:1782488127.064689:0:213384:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298640256 00000100:00100000:3.0:1782488127.064709:0:213384:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 697 00000100:00100000:3.0:1782488127.064718:0:213384:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb574910380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2362:x1869066298640384:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:3.0:1782488127.064791:0:213384:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb574910380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2362:x1869066298640384:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 73us (235us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488127.064798:0:213384:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 697 00000100:00100000:3.0:1782488127.064813:0:213384:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 698 00000100:00100000:3.0:1782488127.064819:0:213384:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb574912a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2363:x1869066298640256:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:3.0:1782488127.064841:0:213384:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb574912a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2363:x1869066298640256:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 23us (257us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488127.064846:0:213384:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 698 00000100:00100000:1.0:1782488127.064914:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298640128 00000100:00100000:1.0:1782488127.064918:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298640128 00000100:00100000:1.0:1782488127.065031:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 901 00000100:00100000:1.0:1782488127.065041:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4525ad500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2362:x1869066298640128:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:1.0:1782488127.065071:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4525ad500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2362:x1869066298640128:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 33us (616us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488127.065078:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 901 00000100:00100000:1.0:1782488127.065100:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298640000 00000100:00100000:1.0:1782488127.065104:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298640000 00000100:00100000:1.0:1782488127.065127:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 554 00000100:00100000:1.0:1782488127.065134:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579adf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2363:x1869066298640000:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:1.0:1782488127.067112:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579adf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2363:x1869066298640000:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 1969us (2574us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488127.067125:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 554 00000100:00080000:1.0:1782488127.067251:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1782488127.067257:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1782488127.067260:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1782488127.108090:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488128.132126:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488129.154261:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0F:1782488130.011731:0:228093:0:(debug.c:654:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanityn test 200: service remains healthy while able to process request ========================================================== 11:35:28 (1782488128) 00010000:00010000:2.0:1782488130.189878:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1782488130.369969:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb573553100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300120704:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1782488130.370039:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488130.370113:0:190933:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300120704 00000100:00100000:1.0:1782488130.370120:0:190933:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300120704 00000100:00100000:1.0:1782488130.370170:0:190933:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 429 00000100:00100000:1.0:1782488130.370190:0:190933:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb573550000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300120704:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1782488130.370224:0:190933:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 30343168 pending 0 free 3806330880 avail 3804233728 00000100:00100000:1.0:1782488130.370271:0:190933:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb573550000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300120704:12345-0@lo:13:osp-pre-1-0.0 Request processed in 81us (231us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488130.370277:0:190933:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 429 00000100:00080000:1.0:1782488130.370299:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1782488130.370333:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb573553100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300120704:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1782488130.370468:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb574913100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3314:1869066300120960:0@lo:400:kworker.0 00000100:00100000:3.0:1782488130.370518:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488130.370538:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1782488130.370551:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45538a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3314:1869066300121088:0@lo:400:kworker.0 00000100:00100000:3.0:1782488130.370587:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488130.370656:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45ae31500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3314:1869066300121472:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1782488130.370676:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488130.370838:0:192482:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300120960 00000100:00100000:1.0:1782488130.370843:0:192482:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300120960 00000100:00100000:1.0:1782488130.370885:0:192482:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 699 00000100:00100000:1.0:1782488130.370892:0:192482:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455389c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3314:x1869066300120960:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488130.370930:0:192482:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455389c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3314:x1869066300120960:12345-0@lo:400:kworker.0 Request processed in 38us (419us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488130.370936:0:192482:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 699 00000100:00100000:1.0:1782488130.370953:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300121088 00000100:00100000:1.0:1782488130.370955:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300121088 00000100:00100000:1.0:1782488130.370978:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 902 00000100:00100000:1.0:1782488130.370983:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455389f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3314:x1869066300121088:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488130.371009:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455389f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3314:x1869066300121088:12345-0@lo:400:kworker.0 Request processed in 26us (421us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488130.371014:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 902 00000100:00100000:1.0:1782488130.371028:0:190932:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300121472 00000100:00100000:1.0:1782488130.371030:0:190932:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300121472 00000100:00100000:1.0:1782488130.371045:0:190932:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 430 00000100:00100000:1.0:1782488130.371050:0:190932:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45ae30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1869066300121472:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1782488130.371067:0:190932:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 38731776 pending 0 free 3825205248 avail 3823108096 00000100:00100000:1.0:1782488130.371099:0:190932:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45ae30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1869066300121472:12345-0@lo:13:osp-pre-0-0.0 Request processed in 49us (423us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488130.371105:0:190932:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 430 00000100:00100000:1.0:1782488130.371126:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb574913800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300120832:0@lo:400:kworker.0 00000100:00100000:1.0:1782488130.371152:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488130.371171:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb455388a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3317:1869066300121344:0@lo:400:kworker.0 00000100:00100000:1.0:1782488130.371189:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488130.371205:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1782488130.371209:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45538ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3317:1869066300121216:0@lo:400:kworker.0 00000100:00100000:1.0:1782488130.371228:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488130.371251:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300120832 00000100:00100000:1.0:1782488130.371254:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300120832 00000100:00100000:1.0:1782488130.371278:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 555 00000100:00100000:1.0:1782488130.371284:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579addc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+19:3317:x1869066300120832:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488130.371316:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579addc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+19:3317:x1869066300120832:12345-0@lo:400:kworker.0 Request processed in 32us (163us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488130.371321:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 555 00000100:00100000:1.0:1782488130.371345:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300121344 00000100:00100000:1.0:1782488130.371349:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300121344 00000100:00100000:1.0:1782488130.371368:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300121216 00000100:00100000:1.0:1782488130.371370:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300121216 00000100:00100000:1.0:1782488130.371389:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 903 00000100:00100000:1.0:1782488130.371395:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb573553100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1869066300121344:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488130.371421:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb573553100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1869066300121344:12345-0@lo:400:kworker.0 Request processed in 27us (231us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488130.371427:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 903 00000100:00100000:1.0:1782488130.371437:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 904 00000100:00100000:1.0:1782488130.371449:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb573553480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3317:x1869066300121216:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488130.371487:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb573553480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3317:x1869066300121216:12345-0@lo:400:kworker.0 Request processed in 38us (258us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488130.371492:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 904 00000100:00080000:3.0:1782488130.371597:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1782488130.371601:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1782488130.371746:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45538ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3317:1869066300121216:0@lo:400:kworker.0 00000100:00100000:1.0:1782488130.371756:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb574913800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300120832:0@lo:400:kworker.0 00000100:00100000:1.0:1782488130.371764:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb455388a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3317:1869066300121344:0@lo:400:kworker.0 00000100:00100000:1.0:1782488130.371799:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45ae31500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3314:1869066300121472:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1782488130.371809:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45538a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3314:1869066300121088:0@lo:400:kworker.0 00000100:00100000:1.0:1782488130.371817:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb574913100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3314:1869066300120960:0@lo:400:kworker.0 00010000:00010000:2.0:1782488131.201724:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1782488132.189744:0:3307:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:3.0:1782488132.189780:0:3307:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:3.0:1782488132.192663:0:3307:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:3.0:1782488132.193531:0:3307:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:3.0:1782488132.193570:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298640512 00000100:00100000:3.0:1782488132.193579:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298640512 00000100:00100000:3.0:1782488132.193651:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 556 00000100:00100000:3.0:1782488132.193661:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579adfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2360:x1869066298640512:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:3.0:1782488132.193729:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579adfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2360:x1869066298640512:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 69us (3998us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488132.193737:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 556 00000100:00100000:3.0:1782488132.193770:0:213225:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298640768 00000100:00100000:3.0:1782488132.193773:0:213225:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298640768 00000100:00100000:3.0:1782488132.193812:0:213225:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298640896 00000100:00100000:3.0:1782488132.193814:0:213225:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298640896 00000100:00100000:3.0:1782488132.193834:0:213225:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 700 00000100:00100000:3.0:1782488132.193840:0:213225:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45ae32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2360:x1869066298640768:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:3.0:1782488132.193865:0:213225:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45ae32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2360:x1869066298640768:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 26us (4084us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488132.193871:0:213225:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 700 00000100:00100000:3.0:1782488132.193882:0:213225:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 701 00000100:00100000:3.0:1782488132.193902:0:213225:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb5491c9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2360:x1869066298640896:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:3.0:1782488132.195244:0:213225:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb5491c9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2360:x1869066298640896:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 1333us (1707us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488132.195257:0:213225:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 701 00000100:00100000:3.0:1782488132.195297:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298640640 00000100:00100000:3.0:1782488132.195303:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298640640 00000100:00100000:3.0:1782488132.195351:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 905 00000100:00100000:3.0:1782488132.195359:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45ae31500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2360:x1869066298640640:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:3.0:1782488132.195399:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45ae31500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2360:x1869066298640640:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 42us (2749us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488132.195406:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 905 00000100:00080000:3.0:1782488132.199193:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1782488132.199203:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1782488132.199206:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1782488132.232747:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488133.250114:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488134.277334:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488135.299812:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1782488135.493701:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45345ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3314:1869066300121984:0@lo:400:kworker.0 00000100:00100000:0.0:1782488135.493776:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488135.493834:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1782488135.493840:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45345e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300121600:0@lo:400:kworker.0 00000100:00100000:0.0:1782488135.493867:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488135.493889:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45345df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3314:1869066300122112:0@lo:400:kworker.0 00000100:00100000:0.0:1782488135.493909:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488135.510683:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300121984 00000100:00100000:0.0:1782488135.510701:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300121984 00000100:00100000:0.0:1782488135.510752:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300122112 00000100:00100000:0.0:1782488135.510755:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300122112 00000100:00100000:0.0:1782488135.510798:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 906 00000100:00100000:0.0:1782488135.510807:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45345ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3314:x1869066300121984:12345-0@lo:400:kworker.0 00000100:00080000:3.0:1782488135.510969:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1782488135.511005:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45345ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3314:x1869066300121984:12345-0@lo:400:kworker.0 Request processed in 199us (17228us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488135.511013:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 906 00000100:00100000:0.0:1782488135.511031:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 907 00000100:00100000:0.0:1782488135.511038:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45345d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3314:x1869066300122112:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488135.511067:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45345d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3314:x1869066300122112:12345-0@lo:400:kworker.0 Request processed in 31us (17157us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488135.511074:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 907 00000100:00100000:0.0:1782488135.511098:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45345ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3317:1869066300121856:0@lo:400:kworker.0 00000100:00100000:0.0:1782488135.511133:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488135.511164:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb48a4c5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3317:1869066300122368:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1782488135.511184:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488135.511218:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1782488135.511224:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45345fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3317:1869066300121728:0@lo:400:kworker.0 00000100:00100000:0.0:1782488135.511280:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488135.511295:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45345dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3317:1869066300122240:0@lo:400:kworker.0 00000100:00100000:0.0:1782488135.511314:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488135.511339:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300122240 00000100:00100000:0.0:1782488135.511342:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300122240 00000100:00100000:0.0:1782488135.511367:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 908 00000100:00100000:0.0:1782488135.511372:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45345f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1869066300122240:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782488135.511486:0:190928:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300121856 00000100:00100000:3.0:1782488135.511490:0:190928:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300121856 00000100:00100000:3.0:1782488135.511514:0:190928:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300121728 00000100:00100000:3.0:1782488135.511516:0:190928:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300121728 00000100:00100000:3.0:1782488135.511531:0:190928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 702 00000100:00100000:3.0:1782488135.511536:0:190928:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45345d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3317:x1869066300121856:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782488135.513534:0:190928:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45345d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3317:x1869066300121856:12345-0@lo:400:kworker.0 Request processed in 1988us (2391us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488135.513549:0:190928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 702 00000100:00100000:3.0:1782488135.513580:0:190928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 703 00000100:00100000:3.0:1782488135.513592:0:190928:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45345e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3317:x1869066300121728:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782488135.513660:0:190928:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45345e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3317:x1869066300121728:12345-0@lo:400:kworker.0 Request processed in 73us (2379us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488135.513680:0:190928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 703 00000100:00100000:0.0:1782488135.513842:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45345f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1869066300122240:12345-0@lo:400:kworker.0 Request processed in 2467us (2524us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488135.513851:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 908 00000100:00100000:0.0:1782488135.513883:0:198596:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300122368 00000100:00100000:0.0:1782488135.513957:0:198596:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300122368 00000100:00100000:0.0:1782488135.513998:0:198596:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 431 00000100:00100000:0.0:1782488135.514004:0:198596:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45345f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1869066300122368:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1782488135.514031:0:198596:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 38731776 pending 0 free 3825205248 avail 3823108096 00000100:00100000:0.0:1782488135.514066:0:198596:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45345f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1869066300122368:12345-0@lo:13:osp-pre-0-0.0 Request processed in 62us (2881us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488135.514072:0:198596:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 431 00000100:00080000:0.0:1782488135.514096:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1782488135.514101:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1782488135.514104:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1782488135.514144:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45345df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3314:1869066300122112:0@lo:400:kworker.0 00000100:00100000:0.0:1782488135.514158:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45345ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3314:1869066300121984:0@lo:400:kworker.0 00000100:00100000:0.0:1782488135.514193:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45345fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3317:1869066300121728:0@lo:400:kworker.0 00000100:00100000:0.0:1782488135.514217:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45345dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3317:1869066300122240:0@lo:400:kworker.0 00000100:00100000:0.0:1782488135.514228:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45345ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3317:1869066300121856:0@lo:400:kworker.0 00000100:00100000:0.0:1782488135.514240:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb48a4c5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3317:1869066300122368:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1782488135.516950:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300121600 00000100:00100000:1.0:1782488135.516964:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300121600 00000100:00100000:1.0:1782488135.517647:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 557 00000100:00100000:1.0:1782488135.517662:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579adc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+19:3314:x1869066300121600:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488135.517788:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579adc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+19:3314:x1869066300121600:12345-0@lo:400:kworker.0 Request processed in 130us (23917us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488135.517796:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 557 00000100:00100000:0.0:1782488135.521672:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45345e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300121600:0@lo:400:kworker.0 00000100:00100000:0.0:1782488135.522856:0:3315:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1782488135.522873:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45345e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3315:1869066300122496:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1782488135.523014:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488135.525137:0:190933:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300122496 00000100:00100000:1.0:1782488135.525153:0:190933:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300122496 00000100:00100000:1.0:1782488135.525213:0:190933:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 432 00000100:00100000:1.0:1782488135.525223:0:190933:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45345ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3315:x1869066300122496:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1782488135.525268:0:190933:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 30343168 pending 0 free 3806330880 avail 3804233728 00000100:00100000:1.0:1782488135.525354:0:190933:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45345ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3315:x1869066300122496:12345-0@lo:13:osp-pre-1-0.0 Request processed in 132us (2339us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488135.525371:0:190933:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 432 00000100:00100000:1.0:1782488135.526232:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45345e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3315:1869066300122496:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1782488136.325559:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1782488137.307771:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:1.0:1782488137.307813:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:1.0:1782488137.307840:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:1.0:1782488137.307870:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:1.0:1782488137.307900:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298641024 00000100:00100000:1.0:1782488137.307909:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298641024 00000100:00100000:1.0:1782488137.307958:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 558 00000100:00100000:1.0:1782488137.307969:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb454603850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2360:x1869066298641024:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:1.0:1782488137.308126:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb454603850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2360:x1869066298641024:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 157us (364us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488137.308135:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 558 00000100:00100000:1.0:1782488137.308171:0:213210:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298641280 00000100:00100000:1.0:1782488137.308174:0:213210:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298641280 00000100:00100000:1.0:1782488137.308207:0:213210:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298641408 00000100:00100000:1.0:1782488137.308211:0:213210:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298641408 00000100:00100000:1.0:1782488137.308233:0:213210:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 704 00000100:00100000:1.0:1782488137.308239:0:213210:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb573551f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2360:x1869066298641280:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:1.0:1782488137.308266:0:213210:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb573551f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2360:x1869066298641280:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 28us (453us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488137.308273:0:213210:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 704 00000100:00100000:1.0:1782488137.308285:0:213210:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 705 00000100:00100000:1.0:1782488137.308291:0:213210:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb573552d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2360:x1869066298641408:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:1.0:1782488137.308314:0:213210:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb573552d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2360:x1869066298641408:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 24us (444us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488137.308327:0:213210:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 705 00000100:00100000:1.0:1782488137.308348:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298641152 00000100:00100000:1.0:1782488137.308351:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298641152 00000100:00100000:1.0:1782488137.308391:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 909 00000100:00100000:1.0:1782488137.308396:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb573553800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2360:x1869066298641152:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:1.0:1782488137.308435:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb573553800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2360:x1869066298641152:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 38us (594us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488137.308442:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 909 00000100:00080000:1.0:1782488137.311101:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1782488137.311111:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1782488137.311114:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1782488137.350308:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488138.369786:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488139.398884:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488140.426441:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1782488140.612476:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb545d69c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3315:1869066300122624:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1782488140.612581:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488140.612695:0:190932:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300122624 00000100:00100000:3.0:1782488140.612702:0:190932:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300122624 00000100:00100000:3.0:1782488140.612755:0:190932:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 433 00000100:00100000:3.0:1782488140.612764:0:190932:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb545d68700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3315:x1869066300122624:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1782488140.612799:0:190932:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 30343168 pending 0 free 3806330880 avail 3804233728 00000100:00100000:3.0:1782488140.612842:0:190932:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb545d68700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3315:x1869066300122624:12345-0@lo:13:osp-pre-1-0.0 Request processed in 78us (261us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488140.612849:0:190932:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 433 00000100:00080000:3.0:1782488140.612869:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1782488140.612902:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb545d69c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3315:1869066300122624:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1782488140.612959:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb545d69c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300122752:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1782488140.612997:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488140.613040:0:198596:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300122752 00000100:00100000:3.0:1782488140.613044:0:198596:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300122752 00000100:00100000:3.0:1782488140.613069:0:198596:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 434 00000100:00100000:3.0:1782488140.613075:0:198596:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb545d69180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300122752:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1782488140.613097:0:198596:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 38731776 pending 0 free 3825205248 avail 3823108096 00000100:00100000:3.0:1782488140.613141:0:198596:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb545d69180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300122752:12345-0@lo:13:osp-pre-0-0.0 Request processed in 66us (142us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488140.613147:0:198596:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 434 00000100:00080000:3.0:1782488140.614404:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1782488140.614453:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb545d69c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300122752:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1782488140.616891:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb545d68380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3314:1869066300123008:0@lo:400:kworker.0 00000100:00100000:3.0:1782488140.616964:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488140.617002:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1782488140.617008:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb453dd2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3314:1869066300123136:0@lo:400:kworker.0 00000100:00100000:3.0:1782488140.617033:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488140.617103:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300123008 00000100:00100000:3.0:1782488140.617110:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300123008 00000100:00100000:3.0:1782488140.617170:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300123136 00000100:00100000:3.0:1782488140.617174:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300123136 00000100:00100000:3.0:1782488140.617209:0:213935:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 910 00000100:00100000:3.0:1782488140.617217:0:213935:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb573531c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-MDT0000_UUID+5:3314:x1869066300123008:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782488140.617269:0:213935:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb573531c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-MDT0000_UUID+5:3314:x1869066300123008:12345-0@lo:400:kworker.0 Request processed in 52us (305us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488140.617276:0:213935:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 910 00000100:00100000:3.0:1782488140.617294:0:213935:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 911 00000100:00100000:3.0:1782488140.617299:0:213935:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb573531180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-OST0000_UUID+5:3314:x1869066300123136:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782488140.617326:0:213935:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb573531180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-OST0000_UUID+5:3314:x1869066300123136:12345-0@lo:400:kworker.0 Request processed in 27us (293us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488140.617331:0:213935:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 911 00000100:00100000:3.0:1782488140.617359:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb453dd0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3316:1869066300123264:0@lo:400:kworker.0 00000100:00100000:3.0:1782488140.617394:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488140.617412:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1782488140.617418:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb545d69c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300122880:0@lo:400:kworker.0 00000100:00100000:3.0:1782488140.617448:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488140.617499:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300122880 00000100:00100000:3.0:1782488140.617503:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300122880 00000100:00100000:3.0:1782488140.617546:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 559 00000100:00100000:3.0:1782488140.617553:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb454602c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+19:3316:x1869066300122880:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782488140.617590:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb454602c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+19:3316:x1869066300122880:12345-0@lo:400:kworker.0 Request processed in 37us (140us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488140.617597:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 559 00000100:00100000:3.0:1782488140.617680:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb453dd2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3314:1869066300123136:0@lo:400:kworker.0 00000100:00100000:3.0:1782488140.617692:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb545d68380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3314:1869066300123008:0@lo:400:kworker.0 00000100:00100000:3.0:1782488140.617724:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb545d69c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300122880:0@lo:400:kworker.0 00000100:00100000:1.0:1782488140.620594:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300123264 00000100:00100000:1.0:1782488140.620626:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300123264 00000100:00100000:1.0:1782488140.620745:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 912 00000100:00100000:1.0:1782488140.620756:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb573531880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3316:x1869066300123264:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488140.620852:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb573531880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3316:x1869066300123264:12345-0@lo:400:kworker.0 Request processed in 98us (3457us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488140.620860:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 912 00000100:00100000:1.0:1782488140.620922:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb453dd0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3316:1869066300123264:0@lo:400:kworker.0 00010000:00010000:0.0:1782488141.444400:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1782488142.426060:0:3307:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:1.0:1782488142.426106:0:3307:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:1.0:1782488142.426176:0:3307:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:1.0:1782488142.426203:0:3307:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:1.0:1782488142.426255:0:213211:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298641920 00000100:00100000:1.0:1782488142.426264:0:213211:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298641920 00000100:00100000:1.0:1782488142.426323:0:213211:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298641792 00000100:00100000:1.0:1782488142.426339:0:213211:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298641792 00000100:00100000:0.0:1782488142.426824:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298641664 00000100:00100000:0.0:1782488142.426830:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298641664 00000100:00100000:0.0:1782488142.426870:0:213935:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 913 00000100:00100000:0.0:1782488142.426879:0:213935:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb574913100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2361:x1869066298641664:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:0.0:1782488142.426976:0:213935:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb574913100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2361:x1869066298641664:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 97us (924us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488142.426984:0:213935:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 913 00000100:00080000:0.0:1782488142.427135:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1782488142.427139:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1782488142.427153:0:213223:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 706 00000100:00100000:0.0:1782488142.427161:0:213223:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb574913800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2361:x1869066298641920:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:0.0:1782488142.427206:0:213223:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb574913800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2361:x1869066298641920:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 47us (1100us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488142.427214:0:213223:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 706 00000100:00100000:0.0:1782488142.427243:0:213223:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 707 00000100:00100000:0.0:1782488142.427259:0:213223:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45538a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2361:x1869066298641792:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:0.0:1782488142.427285:0:213223:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45538a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2361:x1869066298641792:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 28us (1082us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488142.427291:0:213223:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 707 00000100:00100000:0.0:1782488142.427310:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298641536 00000100:00100000:0.0:1782488142.427313:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298641536 00000100:00100000:0.0:1782488142.427338:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 560 00000100:00100000:0.0:1782488142.427344:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb454601850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2361:x1869066298641536:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:0.0:1782488142.432858:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb454601850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2361:x1869066298641536:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 5501us (6669us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488142.432873:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 560 00010000:00010000:0.0:1782488142.471711:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488143.490238:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488144.513998:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488145.542499:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1782488145.731978:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb545d69c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3317:1869066300123392:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1782488145.732066:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488145.732157:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb545d68380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3314:1869066300123520:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1782488145.732197:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488145.732445:0:190933:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300123392 00000100:00100000:3.0:1782488145.732452:0:190933:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300123392 00000100:00100000:3.0:1782488145.732502:0:190933:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300123520 00000100:00100000:3.0:1782488145.732506:0:190933:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300123520 00000100:00100000:3.0:1782488145.732525:0:190933:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 435 00000100:00100000:3.0:1782488145.732533:0:190933:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb453dd0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1869066300123392:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1782488145.732583:0:190933:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 38731776 pending 0 free 3825205248 avail 3823108096 00000100:00100000:3.0:1782488145.732644:0:190933:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb453dd0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1869066300123392:12345-0@lo:13:osp-pre-0-0.0 Request processed in 111us (578us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488145.732652:0:190933:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 435 00000100:00100000:3.0:1782488145.732670:0:190933:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 436 00000100:00100000:3.0:1782488145.734799:0:190933:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb453dd2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1869066300123520:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1782488145.734853:0:190933:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 30343168 pending 0 free 3806330880 avail 3804233728 00000100:00100000:3.0:1782488145.734937:0:190933:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb453dd2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1869066300123520:12345-0@lo:13:osp-pre-1-0.0 Request processed in 154us (2739us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488145.734946:0:190933:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 436 00000100:00080000:3.0:1782488145.735215:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1782488145.735221:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1782488145.735286:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb545d69c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3317:1869066300123392:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1782488145.735332:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb545d68380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3314:1869066300123520:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1782488145.735938:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb455388a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300123648:0@lo:400:kworker.0 00000100:00100000:1.0:1782488145.736008:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488145.736045:0:3315:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1782488145.736051:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb449fb6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3315:1869066300124032:0@lo:400:kworker.0 00000100:00100000:1.0:1782488145.736078:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488145.736107:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300123648 00000100:00100000:1.0:1782488145.736114:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300123648 00000100:00100000:1.0:1782488145.736153:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 561 00000100:00100000:1.0:1782488145.736161:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb454602450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+19:3315:x1869066300123648:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488145.736204:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb454602450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+19:3315:x1869066300123648:12345-0@lo:400:kworker.0 Request processed in 43us (195us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488145.736212:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 561 00000100:00100000:1.0:1782488145.736245:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300124032 00000100:00100000:1.0:1782488145.736249:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300124032 00000100:00100000:1.0:1782488145.736277:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 914 00000100:00100000:1.0:1782488145.736283:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb449fb6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3315:x1869066300124032:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488145.736314:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb449fb6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3315:x1869066300124032:12345-0@lo:400:kworker.0 Request processed in 31us (235us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488145.736320:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 914 00000100:00100000:1.0:1782488145.736369:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb449fb6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3315:1869066300124032:0@lo:400:kworker.0 00000100:00100000:1.0:1782488145.736379:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb455388a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300123648:0@lo:400:kworker.0 00000100:00100000:1.0:1782488145.736404:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb449fb5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3317:1869066300123904:0@lo:400:kworker.0 00000100:00100000:1.0:1782488145.736429:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488145.736447:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1782488145.736452:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45538ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3317:1869066300123776:0@lo:400:kworker.0 00000100:00100000:1.0:1782488145.736488:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488145.736531:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300123904 00000100:00100000:1.0:1782488145.736535:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300123904 00000100:00100000:1.0:1782488145.736555:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300123776 00000100:00100000:1.0:1782488145.736557:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300123776 00000100:00100000:1.0:1782488145.736574:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 915 00000100:00100000:1.0:1782488145.736580:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb449fb6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3317:x1869066300123904:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488145.736648:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb449fb6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3317:x1869066300123904:12345-0@lo:400:kworker.0 Request processed in 68us (218us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488145.736655:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 915 00000100:00100000:1.0:1782488145.736667:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 916 00000100:00100000:0.0:1782488145.736841:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb449fb5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3317:1869066300123904:0@lo:400:kworker.0 00000100:00100000:1.0:1782488145.738356:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455388a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1869066300123776:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488145.738434:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455388a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1869066300123776:12345-0@lo:400:kworker.0 Request processed in 1760us (1944us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488145.738443:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 916 00000100:00100000:0.0:1782488145.738593:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45538ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3317:1869066300123776:0@lo:400:kworker.0 00010000:00010000:0.0:1782488146.572630:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1782488147.541458:0:3307:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:1.0:1782488147.541523:0:3307:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:1.0:1782488147.541550:0:3307:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:1.0:1782488147.541577:0:3307:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:1.0:1782488147.541601:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298642176 00000100:00100000:1.0:1782488147.541622:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298642176 00000100:00100000:1.0:1782488147.541675:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 917 00000100:00100000:1.0:1782488147.541685:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45538ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2361:x1869066298642176:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:1.0:1782488147.541761:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45538ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2361:x1869066298642176:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 77us (312us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488147.541769:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 917 00000100:00100000:1.0:1782488147.541811:0:213227:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298642432 00000100:00100000:1.0:1782488147.541814:0:213227:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298642432 00000100:00100000:1.0:1782488147.541844:0:213227:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298642304 00000100:00100000:1.0:1782488147.541847:0:213227:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298642304 00000100:00100000:1.0:1782488147.541868:0:213227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 708 00000100:00100000:1.0:1782488147.541875:0:213227:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb488519f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2361:x1869066298642432:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:1.0:1782488147.541901:0:213227:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb488519f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2361:x1869066298642432:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 27us (378us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488147.541908:0:213227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 708 00000100:00100000:1.0:1782488147.541918:0:213227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 709 00000100:00100000:1.0:1782488147.541923:0:213227:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb44a0e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2361:x1869066298642304:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:1.0:1782488147.541947:0:213227:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb44a0e6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2361:x1869066298642304:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 23us (369us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488147.541952:0:213227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 709 00000100:00100000:1.0:1782488147.541972:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298642048 00000100:00100000:1.0:1782488147.541975:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298642048 00000100:00100000:1.0:1782488147.541998:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 562 00000100:00100000:1.0:1782488147.542003:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb454600850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2361:x1869066298642048:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:1.0:1782488147.542026:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb454600850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2361:x1869066298642048:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 23us (475us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488147.542033:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 562 00000100:00080000:1.0:1782488147.542179:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1782488147.542182:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1782488147.585864:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488148.612873:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488149.638108:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488150.664554:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1782488150.851226:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45345dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300124160:0@lo:400:kworker.0 00000100:00100000:0.0:1782488150.851318:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488150.851406:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45345d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3315:1869066300124672:0@lo:400:kworker.0 00000100:00100000:0.0:1782488150.851448:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488150.851475:0:3315:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1782488150.851481:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45345df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3315:1869066300124544:0@lo:400:kworker.0 00000100:00100000:0.0:1782488150.851503:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488150.851564:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45345fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300124288:0@lo:400:kworker.0 00000100:00100000:0.0:1782488150.851593:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488150.851636:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45345f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3316:1869066300124800:0@lo:400:kworker.0 00000100:00100000:0.0:1782488150.851658:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488150.851676:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1782488150.851681:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45345ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300124416:0@lo:400:kworker.0 00000100:00100000:0.0:1782488150.851703:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488150.851736:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300124160 00000100:00100000:0.0:1782488150.851744:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300124160 00000100:00100000:0.0:1782488150.851799:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 563 00000100:00100000:0.0:1782488150.851807:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb454603c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+19:3315:x1869066300124160:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488150.851870:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb454603c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+19:3315:x1869066300124160:12345-0@lo:400:kworker.0 Request processed in 63us (551us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488150.851878:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 563 00000100:00100000:0.0:1782488150.851911:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300124672 00000100:00100000:0.0:1782488150.851914:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300124672 00000100:00100000:0.0:1782488150.851938:0:213935:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 918 00000100:00100000:0.0:1782488150.851944:0:213935:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45345c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-OST0000_UUID+5:3315:x1869066300124672:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488150.851975:0:213935:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45345c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-OST0000_UUID+5:3315:x1869066300124672:12345-0@lo:400:kworker.0 Request processed in 31us (527us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488150.851982:0:213935:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 918 00000100:00100000:0.0:1782488150.851992:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300124544 00000100:00100000:0.0:1782488150.851995:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300124544 00000100:00100000:0.0:1782488150.852013:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300124800 00000100:00100000:0.0:1782488150.852016:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300124800 00000100:00100000:0.0:1782488150.852035:0:213935:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 919 00000100:00100000:0.0:1782488150.852041:0:213935:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45345c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-MDT0000_UUID+5:3315:x1869066300124544:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488150.852080:0:213935:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45345c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-MDT0000_UUID+5:3315:x1869066300124544:12345-0@lo:400:kworker.0 Request processed in 38us (575us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488150.852087:0:213935:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 919 00000100:00100000:0.0:1782488150.852099:0:213935:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 920 00000100:00100000:0.0:1782488150.852105:0:213935:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb449fb5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-OST0001_UUID+5:3316:x1869066300124800:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488150.852137:0:213935:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb449fb5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:lustre-MDT0000-lwp-OST0001_UUID+5:3316:x1869066300124800:12345-0@lo:400:kworker.0 Request processed in 31us (477us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488150.852142:0:213935:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 920 00000100:00100000:0.0:1782488150.852163:0:213225:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300124288 00000100:00100000:0.0:1782488150.852167:0:213225:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300124288 00000100:00100000:0.0:1782488150.852199:0:213225:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300124416 00000100:00100000:0.0:1782488150.852203:0:213225:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300124416 00000100:00100000:0.0:1782488150.852223:0:213225:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 710 00000100:00100000:0.0:1782488150.852230:0:213225:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45345e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300124288:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488150.852258:0:213225:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45345e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300124288:12345-0@lo:400:kworker.0 Request processed in 29us (664us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488150.852264:0:213225:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 710 00000100:00100000:0.0:1782488150.852275:0:213225:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 711 00000100:00100000:0.0:1782488150.852280:0:213225:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb459632a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300124416:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488150.852320:0:213225:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb459632a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300124416:12345-0@lo:400:kworker.0 Request processed in 39us (615us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488150.852325:0:213225:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 711 00000100:00080000:0.0:1782488150.852359:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1782488150.852363:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1782488150.852414:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45345ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300124416:0@lo:400:kworker.0 00000100:00100000:0.0:1782488150.852426:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45345fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300124288:0@lo:400:kworker.0 00000100:00100000:0.0:1782488150.852445:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45345f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3316:1869066300124800:0@lo:400:kworker.0 00000100:00100000:0.0:1782488150.852477:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45345df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3315:1869066300124544:0@lo:400:kworker.0 00000100:00100000:0.0:1782488150.852485:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45345dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300124160:0@lo:400:kworker.0 00000100:00100000:0.0:1782488150.852493:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45345d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3315:1869066300124672:0@lo:400:kworker.0 00000100:00100000:0.0:1782488150.853888:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb545d68380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3317:1869066300124928:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1782488150.853919:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb459631f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3314:1869066300125056:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1782488150.861022:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488150.861124:0:190933:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300124928 00000100:00100000:0.0:1782488150.861135:0:190933:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300124928 00000100:00100000:0.0:1782488150.861196:0:190933:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 437 00000100:00100000:0.0:1782488150.861208:0:190933:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45345d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1869066300124928:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1782488150.861254:0:190933:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 38731776 pending 0 free 3825205248 avail 3823108096 00000100:00100000:0.0:1782488150.861344:0:190933:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45345d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1869066300124928:12345-0@lo:13:osp-pre-0-0.0 Request processed in 137us (328us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488150.861352:0:190933:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 437 00000100:00100000:0.0:1782488150.861409:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb545d68380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3317:1869066300124928:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1782488150.863566:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488150.863636:0:190932:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300125056 00000100:00100000:0.0:1782488150.863645:0:190932:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300125056 00000100:00100000:0.0:1782488150.863692:0:190932:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 438 00000100:00100000:0.0:1782488150.863702:0:190932:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb48a4c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1869066300125056:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1782488150.863739:0:190932:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 30343168 pending 0 free 3806330880 avail 3804233728 00000100:00100000:0.0:1782488150.863793:0:190932:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb48a4c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1869066300125056:12345-0@lo:13:osp-pre-1-0.0 Request processed in 92us (233us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488150.863800:0:190932:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 438 00000100:00100000:2.0:1782488150.864228:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb459631f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3314:1869066300125056:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1782488151.681769:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1782488152.661028:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:0.0:1782488152.662034:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:0.0:1782488152.662056:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:0.0:1782488152.662139:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:0.0:1782488152.662161:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298642688 00000100:00100000:0.0:1782488152.662169:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298642688 00000100:00100000:0.0:1782488152.662216:0:213935:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 921 00000100:00100000:0.0:1782488152.662225:0:213935:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45345dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2360:x1869066298642688:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:0.0:1782488152.662304:0:213935:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45345dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2360:x1869066298642688:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 79us (272us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488152.662311:0:213935:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 921 00000100:00100000:0.0:1782488152.662333:0:190928:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298642944 00000100:00100000:0.0:1782488152.662336:0:190928:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298642944 00000100:00100000:0.0:1782488152.662383:0:190928:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298642816 00000100:00100000:0.0:1782488152.662386:0:190928:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298642816 00000100:00100000:0.0:1782488152.662668:0:190928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 712 00000100:00100000:0.0:1782488152.662678:0:190928:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45345df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2361:x1869066298642944:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:0.0:1782488152.662708:0:190928:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45345df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2361:x1869066298642944:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 33us (650us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488152.662715:0:190928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 712 00000100:00100000:0.0:1782488152.662728:0:190928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 713 00000100:00100000:0.0:1782488152.662735:0:190928:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45345f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2361:x1869066298642816:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:0.0:1782488152.662757:0:190928:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45345f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2361:x1869066298642816:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 23us (618us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488152.662763:0:190928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 713 00000100:00080000:0.0:1782488152.662859:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1782488152.662871:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1782488152.668542:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298642560 00000100:00100000:2.0:1782488152.668555:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298642560 00000100:00100000:2.0:1782488152.668639:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 564 00000100:00100000:2.0:1782488152.668649:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb454601450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2360:x1869066298642560:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:2.0:1782488152.668742:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb454601450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2360:x1869066298642560:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 93us (7721us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488152.668749:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 564 00010000:00010000:2.0:1782488152.713052:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488153.730301:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488154.760669:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488155.777786:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1782488155.969899:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45345fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300125184:0@lo:400:kworker.0 00000100:00100000:0.0:1782488155.969992:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488155.970038:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4c4d9e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3315:1869066300125696:0@lo:400:kworker.0 00000100:00100000:0.0:1782488155.970079:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488155.970105:0:3315:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1782488155.970113:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb453459c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3315:1869066300125568:0@lo:400:kworker.0 00000100:00100000:0.0:1782488155.970163:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488155.970191:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45345ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300125312:0@lo:400:kworker.0 00000100:00100000:0.0:1782488155.970226:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488155.970246:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4c4d9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3316:1869066300125824:0@lo:400:kworker.0 00000100:00100000:0.0:1782488155.970277:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488155.970294:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1782488155.970299:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb453458a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300125440:0@lo:400:kworker.0 00000100:00100000:0.0:1782488155.970331:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488155.970905:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300125184 00000100:00100000:0.0:1782488155.970915:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300125184 00000100:00100000:0.0:1782488155.970961:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 565 00000100:00100000:0.0:1782488155.970971:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb454602850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+19:3315:x1869066300125184:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488155.971029:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb454602850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+19:3315:x1869066300125184:12345-0@lo:400:kworker.0 Request processed in 59us (1038us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488155.971036:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 565 00000100:00100000:0.0:1782488155.971089:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300125696 00000100:00100000:0.0:1782488155.971093:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300125696 00000100:00100000:0.0:1782488155.971139:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300125568 00000100:00100000:0.0:1782488155.971142:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300125568 00000100:00100000:0.0:1782488155.971158:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300125824 00000100:00100000:0.0:1782488155.971173:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300125824 00000100:00100000:0.0:1782488155.971218:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 922 00000100:00100000:0.0:1782488155.971226:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb546c3ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3315:x1869066300125696:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488155.971276:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb546c3ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3315:x1869066300125696:12345-0@lo:400:kworker.0 Request processed in 51us (1195us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488155.971283:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 922 00000100:00100000:0.0:1782488155.971296:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 923 00000100:00100000:0.0:1782488155.971301:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb54815d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3315:x1869066300125568:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488155.971328:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb54815d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3315:x1869066300125568:12345-0@lo:400:kworker.0 Request processed in 27us (1164us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488155.971334:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 923 00000100:00100000:0.0:1782488155.971344:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 924 00000100:00100000:0.0:1782488155.971349:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb44fb97b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3316:x1869066300125824:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488155.971391:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb44fb97b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3316:x1869066300125824:12345-0@lo:400:kworker.0 Request processed in 41us (1112us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488155.971397:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 924 00000100:00100000:0.0:1782488155.971419:0:213210:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300125312 00000100:00100000:0.0:1782488155.971423:0:213210:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300125312 00000100:00100000:0.0:1782488155.971465:0:213210:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300125440 00000100:00100000:0.0:1782488155.971469:0:213210:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300125440 00000100:00100000:0.0:1782488155.971487:0:213210:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 714 00000100:00100000:0.0:1782488155.971493:0:213210:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb44fb97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300125312:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488155.971523:0:213210:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb44fb97800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300125312:12345-0@lo:400:kworker.0 Request processed in 30us (1296us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488155.971529:0:213210:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 714 00000100:00100000:0.0:1782488155.971539:0:213210:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 715 00000100:00100000:0.0:1782488155.971544:0:213210:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb44fb95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300125440:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488155.971581:0:213210:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb44fb95c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300125440:12345-0@lo:400:kworker.0 Request processed in 37us (1249us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488155.971587:0:213210:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 715 00000100:00080000:0.0:1782488155.971659:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1782488155.971663:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1782488155.971697:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb453458a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300125440:0@lo:400:kworker.0 00000100:00100000:0.0:1782488155.971709:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45345ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300125312:0@lo:400:kworker.0 00000100:00100000:0.0:1782488155.971719:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4c4d9c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3316:1869066300125824:0@lo:400:kworker.0 00000100:00100000:0.0:1782488155.971753:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb453459c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3315:1869066300125568:0@lo:400:kworker.0 00000100:00100000:0.0:1782488155.971762:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45345fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300125184:0@lo:400:kworker.0 00000100:00100000:0.0:1782488155.971773:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4c4d9e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3315:1869066300125696:0@lo:400:kworker.0 00000100:00100000:1.0:1782488155.972141:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1782488155.972163:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb49058ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300125952:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1782488155.972231:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488155.972281:0:198596:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300125952 00000100:00100000:1.0:1782488155.972286:0:198596:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300125952 00000100:00100000:1.0:1782488155.972316:0:198596:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 439 00000100:00100000:1.0:1782488155.972323:0:198596:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb560a06300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300125952:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1782488155.972539:0:198596:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 38731776 pending 0 free 3825205248 avail 3823108096 00000100:00100000:1.0:1782488155.972597:0:198596:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb560a06300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300125952:12345-0@lo:13:osp-pre-0-0.0 Request processed in 274us (366us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488155.972604:0:198596:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 439 00000100:00100000:1.0:1782488155.972662:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb49058ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300125952:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1782488155.984857:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4c4d9e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3314:1869066300126080:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1782488155.984928:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488155.984983:0:190933:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300126080 00000100:00100000:0.0:1782488155.984991:0:190933:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300126080 00000100:00100000:0.0:1782488155.985037:0:190933:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 440 00000100:00100000:0.0:1782488155.985045:0:190933:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4c4d9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1869066300126080:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1782488155.985080:0:190933:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 30343168 pending 0 free 3806330880 avail 3804233728 00000100:00100000:0.0:1782488155.985123:0:190933:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4c4d9c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1869066300126080:12345-0@lo:13:osp-pre-1-0.0 Request processed in 78us (195us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488155.985130:0:190933:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 440 00000100:00100000:0.0:1782488155.985747:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4c4d9e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3314:1869066300126080:0@lo:13:osp-pre-1-0.0 00010000:00010000:1.0:1782488156.813630:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1782488157.779910:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:0.0:1782488157.779954:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:0.0:1782488157.779977:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:0.0:1782488157.780020:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:0.0:1782488157.780057:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298643200 00000100:00100000:0.0:1782488157.780066:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298643200 00000100:00100000:0.0:1782488157.780117:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 925 00000100:00100000:0.0:1782488157.780126:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4c4d9e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2361:x1869066298643200:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:0.0:1782488157.780462:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4c4d9e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2361:x1869066298643200:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 336us (561us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488157.780470:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 925 00000100:00100000:0.0:1782488157.780494:0:213223:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298643456 00000100:00100000:0.0:1782488157.780497:0:213223:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298643456 00000100:00100000:0.0:1782488157.780524:0:213223:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298643328 00000100:00100000:0.0:1782488157.780527:0:213223:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298643328 00000100:00100000:0.0:1782488157.780544:0:213223:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 716 00000100:00100000:0.0:1782488157.780550:0:213223:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45345fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2361:x1869066298643456:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:0.0:1782488157.780573:0:213223:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45345fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2361:x1869066298643456:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 23us (619us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488157.780579:0:213223:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 716 00000100:00100000:0.0:1782488157.780588:0:213223:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 717 00000100:00100000:0.0:1782488157.780593:0:213223:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45345ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2361:x1869066298643328:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:0.0:1782488157.780627:0:213223:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45345ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2361:x1869066298643328:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 33us (607us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488157.780633:0:213223:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 717 00000100:00100000:0.0:1782488157.780648:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298643072 00000100:00100000:0.0:1782488157.780650:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298643072 00000100:00100000:0.0:1782488157.780677:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 566 00000100:00100000:0.0:1782488157.780683:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb454600050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2361:x1869066298643072:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:0.0:1782488157.781822:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb454600050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2361:x1869066298643072:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 1134us (1839us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488157.781833:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 566 00000100:00080000:0.0:1782488157.783054:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1782488157.783061:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1782488157.826329:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488158.849764:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488159.874253:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488160.901803:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1782488161.090231:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1782488161.090252:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb545d69c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3314:1869066300126208:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1782488161.090326:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488161.090440:0:190932:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300126208 00000100:00100000:3.0:1782488161.090448:0:190932:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300126208 00000100:00100000:3.0:1782488161.090522:0:190932:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 441 00000100:00100000:3.0:1782488161.090532:0:190932:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb545d68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1869066300126208:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1782488161.090706:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb48a4c5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300126336:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1782488161.090749:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488161.090804:0:190933:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300126336 00000100:00100000:2.0:1782488161.090807:0:190933:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300126336 00000100:00100000:2.0:1782488161.090833:0:190933:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 442 00000100:00100000:2.0:1782488161.090842:0:190933:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb48a4c5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300126336:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1782488161.090874:0:190933:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 30343168 pending 0 free 3806330880 avail 3804233728 00000100:00100000:2.0:1782488161.090922:0:190933:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb48a4c5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300126336:12345-0@lo:13:osp-pre-1-0.0 Request processed in 80us (172us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488161.090929:0:190933:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 442 00000100:00100000:2.0:1782488161.090992:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb48a4c5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300126336:0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1782488161.091495:0:190932:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 38731776 pending 0 free 3825205248 avail 3823108096 00000100:00100000:3.0:1782488161.091628:0:190932:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb545d68380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1869066300126208:12345-0@lo:13:osp-pre-0-0.0 Request processed in 1097us (1302us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488161.091636:0:190932:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 441 00000100:00080000:3.0:1782488161.091895:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1782488161.091900:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1782488161.091918:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300126464:0@lo:400:kworker.0 00000100:00100000:3.0:1782488161.091955:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488161.092265:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1782488161.092287:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45013aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3317:1869066300126848:0@lo:400:kworker.0 00000100:00100000:3.0:1782488161.092320:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488161.092358:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300126464 00000100:00100000:3.0:1782488161.092363:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300126464 00000100:00100000:3.0:1782488161.092410:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 567 00000100:00100000:3.0:1782488161.092420:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb454600c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+19:3317:x1869066300126464:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782488161.092465:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb454600c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+19:3317:x1869066300126464:12345-0@lo:400:kworker.0 Request processed in 47us (510us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488161.092473:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 567 00000100:00100000:2.0:1782488161.092761:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb459637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3314:1869066300126592:0@lo:400:kworker.0 00000100:00100000:2.0:1782488161.092794:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488161.092844:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb545d69c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3314:1869066300126208:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1782488161.092863:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1782488161.092870:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb459635880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3314:1869066300126720:0@lo:400:kworker.0 00000100:00100000:2.0:1782488161.092896:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488161.093323:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300126848 00000100:00100000:3.0:1782488161.093329:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300126848 00000100:00100000:3.0:1782488161.093367:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300126592 00000100:00100000:3.0:1782488161.093371:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300126592 00000100:00100000:3.0:1782488161.093396:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300126720 00000100:00100000:3.0:1782488161.093400:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300126720 00000100:00100000:3.0:1782488161.093424:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 926 00000100:00100000:3.0:1782488161.093432:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb450139500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1869066300126848:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782488161.093473:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb450139500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3317:x1869066300126848:12345-0@lo:400:kworker.0 Request processed in 42us (1153us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488161.093633:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 927 00000100:00100000:2.0:1782488161.093643:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb48a4c5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3314:x1869066300126592:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488161.093707:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb48a4c5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3314:x1869066300126592:12345-0@lo:400:kworker.0 Request processed in 66us (912us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488161.093714:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 927 00000100:00100000:2.0:1782488161.093736:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 928 00000100:00100000:2.0:1782488161.093745:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb545d69c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3314:x1869066300126720:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488161.093780:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb545d69c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3314:x1869066300126720:12345-0@lo:400:kworker.0 Request processed in 36us (882us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488161.093787:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 928 00000100:00100000:2.0:1782488161.093832:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb459635880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3314:1869066300126720:0@lo:400:kworker.0 00000100:00100000:2.0:1782488161.093849:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb459637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3314:1869066300126592:0@lo:400:kworker.0 00000100:00100000:3.0:1782488161.094949:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 926 00000100:00100000:3.0:1782488161.095307:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45013aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3317:1869066300126848:0@lo:400:kworker.0 00000100:00100000:3.0:1782488161.095323:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300126464:0@lo:400:kworker.0 00010000:00010000:2.0:1782488161.924975:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1782488162.905163:0:3308:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:1.0:1782488162.905219:0:3308:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:1.0:1782488162.905266:0:3308:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:1.0:1782488162.905322:0:3308:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:1.0:1782488162.905358:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298643712 00000100:00100000:1.0:1782488162.905368:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298643712 00000100:00100000:1.0:1782488162.905427:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 929 00000100:00100000:1.0:1782488162.905439:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb49058ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2362:x1869066298643712:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:1.0:1782488162.905520:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb49058ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2362:x1869066298643712:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 82us (363us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488162.905529:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 929 00000100:00100000:1.0:1782488162.905579:0:213227:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298643968 00000100:00100000:1.0:1782488162.905584:0:213227:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298643968 00000100:00100000:1.0:1782488162.905652:0:213227:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298643840 00000100:00100000:1.0:1782488162.905656:0:213227:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298643840 00000100:00100000:1.0:1782488162.905681:0:213227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 718 00000100:00100000:1.0:1782488162.905701:0:213227:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb450369880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2362:x1869066298643968:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:1.0:1782488162.905731:0:213227:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb450369880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2362:x1869066298643968:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 31us (513us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488162.905738:0:213227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 718 00000100:00100000:1.0:1782488162.905755:0:213227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 719 00000100:00100000:1.0:1782488162.905761:0:213227:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb452581c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2362:x1869066298643840:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:1.0:1782488162.905784:0:213227:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb452581c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2362:x1869066298643840:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 23us (463us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488162.905791:0:213227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 719 00000100:00080000:1.0:1782488162.905940:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1782488162.905945:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1782488162.906115:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298643584 00000100:00100000:0.0:1782488162.906120:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298643584 00000100:00100000:0.0:1782488162.906164:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 568 00000100:00100000:0.0:1782488162.906174:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb454601050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2362:x1869066298643584:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:0.0:1782488162.906225:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb454601050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2362:x1869066298643584:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 53us (960us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488162.906233:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 568 00010000:00010000:0.0:1782488162.946732:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488163.977433:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488165.008978:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488166.020883:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1782488166.209887:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45013b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300127232:0@lo:400:kworker.0 00000100:00100000:3.0:1782488166.209917:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45013aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3317:1869066300127104:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1782488166.225676:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488166.225771:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442930e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3314:1869066300127744:0@lo:400:kworker.0 00000100:00100000:2.0:1782488166.225814:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488166.225834:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1782488166.225841:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45013a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3314:1869066300127360:0@lo:400:kworker.0 00000100:00100000:2.0:1782488166.225861:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488166.225879:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442931c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3314:1869066300127872:0@lo:400:kworker.0 00000100:00100000:2.0:1782488166.225898:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488166.225930:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300127232 00000100:00100000:2.0:1782488166.225937:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300127232 00000100:00100000:2.0:1782488166.226040:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 569 00000100:00100000:2.0:1782488166.226049:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb454600450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+19:3314:x1869066300127232:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488166.226102:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb454600450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+19:3314:x1869066300127232:12345-0@lo:400:kworker.0 Request processed in 54us (433us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488166.226110:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 569 00000100:00100000:2.0:1782488166.226144:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300127744 00000100:00100000:2.0:1782488166.226148:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300127744 00000100:00100000:2.0:1782488166.226185:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300127872 00000100:00100000:2.0:1782488166.226188:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300127872 00000100:00100000:2.0:1782488166.226206:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 930 00000100:00100000:2.0:1782488166.226212:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb459635880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3314:x1869066300127744:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488166.226242:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb459635880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3314:x1869066300127744:12345-0@lo:400:kworker.0 Request processed in 30us (427us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488166.226247:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 930 00000100:00100000:2.0:1782488166.226257:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 931 00000100:00100000:2.0:1782488166.226262:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb547781880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3314:x1869066300127872:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488166.226282:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb547781880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3314:x1869066300127872:12345-0@lo:400:kworker.0 Request processed in 21us (383us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488166.226287:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 931 00000100:00100000:2.0:1782488166.226302:0:213225:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300127360 00000100:00100000:2.0:1782488166.226306:0:213225:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300127360 00000100:00100000:2.0:1782488166.226339:0:213225:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 720 00000100:00100000:2.0:1782488166.226345:0:213225:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb459631f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3314:x1869066300127360:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488166.226373:0:213225:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb459631f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3314:x1869066300127360:12345-0@lo:400:kworker.0 Request processed in 29us (510us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488166.226379:0:213225:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 720 00000100:00080000:2.0:1782488166.226406:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1782488166.229299:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45013a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3314:1869066300127360:0@lo:400:kworker.0 00000100:00100000:2.0:1782488166.231190:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442931c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3314:1869066300127872:0@lo:400:kworker.0 00000100:00100000:2.0:1782488166.231359:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442930e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3314:1869066300127744:0@lo:400:kworker.0 00000100:00100000:2.0:1782488166.231376:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45013b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300127232:0@lo:400:kworker.0 00000100:00100000:2.0:1782488166.231447:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45013bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300127488:0@lo:400:kworker.0 00000100:00100000:2.0:1782488166.231513:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488166.231601:0:213693:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300127488 00000100:00100000:2.0:1782488166.231900:0:213693:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300127488 00000100:00100000:2.0:1782488166.231998:0:213693:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 721 00000100:00100000:2.0:1782488166.232008:0:213693:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45013b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300127488:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488166.232063:0:213693:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45013b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300127488:12345-0@lo:400:kworker.0 Request processed in 56us (549us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488166.232070:0:213693:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 721 00000100:00080000:2.0:1782488166.232105:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1782488166.232126:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45013bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300127488:0@lo:400:kworker.0 00000100:00100000:3.0:1782488166.236696:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488166.239685:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb459637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3317:1869066300126976:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1782488166.239796:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488166.239830:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3317:1869066300127616:0@lo:400:kworker.0 00000100:00100000:1.0:1782488166.239885:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488166.239931:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300127616 00000100:00100000:1.0:1782488166.239938:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300127616 00000100:00100000:1.0:1782488166.239996:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 932 00000100:00100000:1.0:1782488166.240006:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb452582300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1869066300127616:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488166.240058:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb452582300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3317:x1869066300127616:12345-0@lo:400:kworker.0 Request processed in 53us (171us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488166.240065:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 932 00000100:00100000:3.0:1782488166.240088:0:190932:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300127104 00000100:00100000:3.0:1782488166.240093:0:190932:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300127104 00000100:00100000:1.0:1782488166.240109:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452cca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3317:1869066300127616:0@lo:400:kworker.0 00000100:00100000:3.0:1782488166.240120:0:190932:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300126976 00000100:00100000:3.0:1782488166.240134:0:190932:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300126976 00000100:00100000:3.0:1782488166.240149:0:190932:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 443 00000100:00100000:3.0:1782488166.240155:0:190932:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb442930000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1869066300127104:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1782488166.240195:0:190932:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 30343168 pending 0 free 3806330880 avail 3804233728 00000100:00100000:3.0:1782488166.240236:0:190932:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb442930000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1869066300127104:12345-0@lo:13:osp-pre-1-0.0 Request processed in 81us (3547us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488166.240243:0:190932:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 443 00000100:00100000:3.0:1782488166.240255:0:190932:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 444 00000100:00100000:3.0:1782488166.240261:0:190932:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb452583100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1869066300126976:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1782488166.240281:0:190932:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 38731776 pending 0 free 3825205248 avail 3823108096 00000100:00100000:3.0:1782488166.240307:0:190932:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb452583100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1869066300126976:12345-0@lo:13:osp-pre-0-0.0 Request processed in 45us (511us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488166.240312:0:190932:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 444 00000100:00100000:1.0:1782488166.243331:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45013aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3317:1869066300127104:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1782488166.243384:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb459637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3317:1869066300126976:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1782488167.043042:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1782488168.048259:0:3308:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:2.0:1782488168.048292:0:3308:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:2.0:1782488168.048325:0:3308:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:2.0:1782488168.048352:0:3308:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:2.0:1782488168.048379:0:190928:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298644480 00000100:00100000:2.0:1782488168.048387:0:190928:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298644480 00000100:00100000:2.0:1782488168.048441:0:190928:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298644352 00000100:00100000:2.0:1782488168.048443:0:190928:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298644352 00000100:00100000:2.0:1782488168.048467:0:190928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 722 00000100:00100000:2.0:1782488168.048478:0:190928:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45013a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2361:x1869066298644480:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:2.0:1782488168.048599:0:190928:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45013a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2361:x1869066298644480:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 123us (305us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488168.048635:0:190928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 722 00000100:00100000:2.0:1782488168.048654:0:190928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 723 00000100:00100000:2.0:1782488168.048661:0:190928:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45013aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2361:x1869066298644352:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:2.0:1782488168.048682:0:190928:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45013aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2361:x1869066298644352:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 23us (330us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488168.048688:0:190928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 723 00000100:00100000:2.0:1782488168.048704:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298644096 00000100:00100000:2.0:1782488168.048708:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298644096 00000100:00100000:2.0:1782488168.048728:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 570 00000100:00100000:2.0:1782488168.048733:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb454602050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2361:x1869066298644096:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:2.0:1782488168.048754:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb454602050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2361:x1869066298644096:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 22us (429us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488168.048759:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 570 00000100:00080000:2.0:1782488168.048783:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1782488168.048787:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1782488168.049691:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298644224 00000100:00100000:3.0:1782488168.049699:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298644224 00000100:00100000:3.0:1782488168.049754:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 933 00000100:00100000:3.0:1782488168.049762:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45013bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2361:x1869066298644224:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:3.0:1782488168.049813:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45013bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2361:x1869066298644224:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 53us (1564us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488168.049819:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 933 00010000:00010000:3.0:1782488168.068591:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488169.090880:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488170.120196:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488171.138852:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1782488171.329716:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1782488171.329737:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452582a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3317:1869066300128000:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1782488171.329794:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488171.335563:0:198596:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300128000 00000100:00100000:2.0:1782488171.336097:0:198596:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300128000 00000100:00100000:2.0:1782488171.336174:0:198596:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 445 00000100:00100000:2.0:1782488171.336183:0:198596:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb452580000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1869066300128000:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1782488171.336280:0:198596:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 30343168 pending 0 free 3806330880 avail 3804233728 00000100:00100000:2.0:1782488171.336357:0:198596:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb452580000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1869066300128000:12345-0@lo:13:osp-pre-1-0.0 Request processed in 174us (6561us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488171.336372:0:198596:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 445 00000100:00100000:2.0:1782488171.336424:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1782488171.336493:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb545d91500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300128128:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1782488171.336526:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1782488171.337529:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1782488171.337543:0:190932:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300128128 00000100:00100000:2.0:1782488171.337549:0:190932:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300128128 00000100:00100000:2.0:1782488171.337655:0:190932:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 446 00000100:00100000:2.0:1782488171.337663:0:190932:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb545d90380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300128128:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1782488171.337687:0:190932:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 38731776 pending 0 free 3825205248 avail 3823108096 00000100:00100000:2.0:1782488171.337727:0:190932:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb545d90380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300128128:12345-0@lo:13:osp-pre-0-0.0 Request processed in 65us (1200us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488171.337733:0:190932:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 446 00000100:00100000:2.0:1782488171.337772:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452582a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3317:1869066300128000:0@lo:13:osp-pre-1-0.0 00000100:00080000:2.0:1782488171.337849:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1782488171.337871:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb545d91500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300128128:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1782488171.342406:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452582680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3315:1869066300128384:0@lo:400:kworker.0 00000100:00100000:1.0:1782488171.342751:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488171.342801:0:3315:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1782488171.342806:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452583800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300128256:0@lo:400:kworker.0 00000100:00100000:1.0:1782488171.342832:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488171.342863:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452582a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3316:1869066300128512:0@lo:400:kworker.0 00000100:00100000:1.0:1782488171.342885:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488171.342900:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1782488171.342904:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb459637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3316:1869066300128640:0@lo:400:kworker.0 00000100:00100000:1.0:1782488171.342954:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488171.342981:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300128384 00000100:00100000:1.0:1782488171.342988:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300128384 00000100:00100000:1.0:1782488171.343024:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300128512 00000100:00100000:1.0:1782488171.343026:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300128512 00000100:00100000:1.0:1782488171.343046:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300128640 00000100:00100000:1.0:1782488171.343048:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300128640 00000100:00100000:1.0:1782488171.343065:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 934 00000100:00100000:1.0:1782488171.343072:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb452cca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3315:x1869066300128384:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488171.343132:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb452cca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3315:x1869066300128384:12345-0@lo:400:kworker.0 Request processed in 59us (380us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488171.343139:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 934 00000100:00100000:1.0:1782488171.343156:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 935 00000100:00100000:1.0:1782488171.343174:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4bb705f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3316:x1869066300128512:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488171.343213:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4bb705f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3316:x1869066300128512:12345-0@lo:400:kworker.0 Request processed in 39us (326us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488171.343218:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 935 00000100:00100000:1.0:1782488171.343230:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 936 00000100:00100000:1.0:1782488171.343236:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4b47c4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3316:x1869066300128640:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488171.343257:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4b47c4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3316:x1869066300128640:12345-0@lo:400:kworker.0 Request processed in 22us (303us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488171.343263:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 936 00000100:00100000:1.0:1782488171.343342:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300128256 00000100:00100000:1.0:1782488171.343346:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300128256 00000100:00100000:1.0:1782488171.343367:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 571 00000100:00100000:1.0:1782488171.343372:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb454603450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+19:3315:x1869066300128256:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488171.343397:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb454603450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+19:3315:x1869066300128256:12345-0@lo:400:kworker.0 Request processed in 25us (564us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488171.343402:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 571 00000100:00100000:1.0:1782488171.343470:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452583800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300128256:0@lo:400:kworker.0 00000100:00100000:1.0:1782488171.343481:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452582680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3315:1869066300128384:0@lo:400:kworker.0 00000100:00100000:1.0:1782488171.343506:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb459637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3316:1869066300128640:0@lo:400:kworker.0 00000100:00100000:1.0:1782488171.343514:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452582a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3316:1869066300128512:0@lo:400:kworker.0 00010000:00010000:0.0:1782488172.177790:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1782488173.142639:0:3308:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:2.0:1782488173.142672:0:3308:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:2.0:1782488173.142702:0:3308:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:0.0:1782488173.142703:0:213210:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298644864 00000100:00100000:0.0:1782488173.142712:0:213210:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298644864 00000100:00100000:0.0:1782488173.143067:0:213210:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 724 00000100:00100000:0.0:1782488173.143078:0:213210:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb545d91500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2360:x1869066298644864:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:0.0:1782488173.143166:0:213210:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb545d91500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2360:x1869066298644864:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 90us (492us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488173.143175:0:213210:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 724 00000100:00100000:2.0:1782488173.144405:0:3308:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:2.0:1782488173.144434:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298644608 00000100:00100000:2.0:1782488173.144440:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298644608 00000100:00080000:3.0:1782488173.144474:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1782488173.144544:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 572 00000100:00100000:2.0:1782488173.144553:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb454601c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2360:x1869066298644608:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:2.0:1782488173.144625:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb454601c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2360:x1869066298644608:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 73us (1995us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488173.144640:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 572 00000100:00100000:1.0:1782488173.147180:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298644736 00000100:00100000:1.0:1782488173.147261:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298644736 00000100:00100000:1.0:1782488173.147322:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 937 00000100:00100000:1.0:1782488173.147332:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb545d90700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2360:x1869066298644736:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:1.0:1782488173.147494:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb545d90700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2360:x1869066298644736:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 162us (4788us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488173.147502:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 937 00000100:00100000:1.0:1782488173.147531:0:200304:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298644992 00000100:00100000:1.0:1782488173.147535:0:200304:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298644992 00000100:00100000:1.0:1782488173.147583:0:200304:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 725 00000100:00100000:1.0:1782488173.147589:0:200304:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb44fa0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2360:x1869066298644992:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:1.0:1782488173.147638:0:200304:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb44fa0e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2360:x1869066298644992:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 49us (3238us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488173.147646:0:200304:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 725 00000100:00080000:3.0:1782488173.148785:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1782488173.193510:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488174.214426:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488175.233801:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488176.268784:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1782488176.451742:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4b47c6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300128768:0@lo:400:kworker.0 00000100:00100000:1.0:1782488176.451820:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488176.452293:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452583800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3314:1869066300129280:0@lo:400:kworker.0 00000100:00100000:1.0:1782488176.452327:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488176.452357:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1782488176.452364:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb459637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3314:1869066300128896:0@lo:400:kworker.0 00000100:00100000:1.0:1782488176.452389:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488176.452407:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb5589e6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3314:1869066300129408:0@lo:400:kworker.0 00000100:00100000:1.0:1782488176.452426:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488176.452453:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452582a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300129024:0@lo:400:kworker.0 00000100:00100000:1.0:1782488176.452476:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488176.452544:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1782488176.452550:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452582680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3316:1869066300129152:0@lo:400:kworker.0 00000100:00100000:1.0:1782488176.452588:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488176.452667:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300128768 00000100:00100000:1.0:1782488176.452674:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300128768 00000100:00100000:1.0:1782488176.452723:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 573 00000100:00100000:1.0:1782488176.452732:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb545d1b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+19:3314:x1869066300128768:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488176.452784:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb545d1b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+19:3314:x1869066300128768:12345-0@lo:400:kworker.0 Request processed in 53us (964us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488176.452791:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 573 00000100:00100000:1.0:1782488176.452826:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300129280 00000100:00100000:1.0:1782488176.452830:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300129280 00000100:00100000:1.0:1782488176.452855:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300129408 00000100:00100000:1.0:1782488176.452857:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300129408 00000100:00100000:1.0:1782488176.454236:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300129152 00000100:00100000:1.0:1782488176.454242:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300129152 00000100:00100000:1.0:1782488176.454266:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 938 00000100:00100000:1.0:1782488176.454274:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb5589e4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3314:x1869066300129280:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488176.454341:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb5589e4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3314:x1869066300129280:12345-0@lo:400:kworker.0 Request processed in 67us (2012us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488176.454348:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 938 00000100:00100000:1.0:1782488176.454367:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 939 00000100:00100000:1.0:1782488176.454374:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb451b2aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3314:x1869066300129408:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488176.454411:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb451b2aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3314:x1869066300129408:12345-0@lo:400:kworker.0 Request processed in 37us (1983us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488176.454418:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 939 00000100:00100000:1.0:1782488176.454436:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 940 00000100:00100000:1.0:1782488176.454445:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4a57b7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1869066300129152:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488176.454480:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4a57b7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1869066300129152:12345-0@lo:400:kworker.0 Request processed in 37us (1890us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488176.454499:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 940 00000100:00100000:1.0:1782488176.454528:0:213223:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300128896 00000100:00100000:1.0:1782488176.454532:0:213223:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300128896 00000100:00100000:1.0:1782488176.454580:0:213223:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300129024 00000100:00100000:1.0:1782488176.454583:0:213223:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300129024 00000100:00100000:1.0:1782488176.454639:0:213223:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 726 00000100:00100000:1.0:1782488176.454647:0:213223:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb451b28000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3314:x1869066300128896:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488176.454703:0:213223:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb451b28000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3314:x1869066300128896:12345-0@lo:400:kworker.0 Request processed in 57us (2312us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488176.454710:0:213223:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 726 00000100:00100000:1.0:1782488176.454723:0:213223:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 727 00000100:00100000:1.0:1782488176.454729:0:213223:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb451b29f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300129024:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488176.454759:0:213223:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb451b29f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300129024:12345-0@lo:400:kworker.0 Request processed in 30us (2282us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488176.454765:0:213223:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 727 00000100:00100000:1.0:1782488176.454820:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb459637b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3314:1869066300128896:0@lo:400:kworker.0 00000100:00100000:1.0:1782488176.454833:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb5589e6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3314:1869066300129408:0@lo:400:kworker.0 00000100:00100000:1.0:1782488176.454844:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4b47c6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300128768:0@lo:400:kworker.0 00000100:00100000:1.0:1782488176.454855:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452583800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3314:1869066300129280:0@lo:400:kworker.0 00000100:00100000:1.0:1782488176.455170:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb44fa0df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300129536:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1782488176.455210:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488176.455244:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452582680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3316:1869066300129152:0@lo:400:kworker.0 00000100:00100000:1.0:1782488176.455255:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452582a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300129024:0@lo:400:kworker.0 00000100:00100000:1.0:1782488176.455282:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1782488176.455289:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4a57b4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300129664:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1782488176.455312:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1782488176.455364:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1782488176.455371:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1782488176.455396:0:190932:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300129536 00000100:00100000:1.0:1782488176.455401:0:190932:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300129536 00000100:00100000:1.0:1782488176.455435:0:190932:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300129664 00000100:00100000:1.0:1782488176.455439:0:190932:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300129664 00000100:00100000:1.0:1782488176.455462:0:190932:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 447 00000100:00100000:1.0:1782488176.455471:0:190932:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4a57b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300129536:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1782488176.455500:0:190932:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 38731776 pending 0 free 3825205248 avail 3823108096 00000100:00100000:1.0:1782488176.455543:0:190932:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4a57b6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300129536:12345-0@lo:13:osp-pre-0-0.0 Request processed in 73us (333us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488176.455551:0:190932:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 447 00000100:00100000:1.0:1782488176.455570:0:190932:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 448 00000100:00100000:1.0:1782488176.455578:0:190932:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4a57b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300129664:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1782488176.455599:0:190932:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 30343168 pending 0 free 3806330880 avail 3804233728 00000100:00100000:1.0:1782488176.455664:0:190932:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4a57b5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300129664:12345-0@lo:13:osp-pre-1-0.0 Request processed in 86us (348us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488176.455671:0:190932:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 448 00000100:00100000:1.0:1782488176.455726:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4a57b4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300129664:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1782488176.455739:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb44fa0df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300129536:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1782488177.283097:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1782488178.268785:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:1.0:1782488178.268828:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:1.0:1782488178.268849:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:1.0:1782488178.268882:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:1.0:1782488178.268911:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298645120 00000100:00100000:1.0:1782488178.268920:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298645120 00000100:00100000:1.0:1782488178.268965:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 574 00000100:00100000:1.0:1782488178.268974:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb545d19450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2360:x1869066298645120:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:1.0:1782488178.269042:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb545d19450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2360:x1869066298645120:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 69us (267us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488178.269050:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 574 00000100:00100000:1.0:1782488178.269103:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298645248 00000100:00100000:1.0:1782488178.269107:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298645248 00000100:00100000:1.0:1782488178.269135:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 941 00000100:00100000:1.0:1782488178.269142:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4a57b4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2361:x1869066298645248:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:1.0:1782488178.269723:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4a57b4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2361:x1869066298645248:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 580us (893us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488178.269731:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 941 00000100:00100000:1.0:1782488178.269755:0:213227:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298645376 00000100:00100000:1.0:1782488178.269759:0:213227:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298645376 00000100:00100000:1.0:1782488178.269793:0:213227:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298645504 00000100:00100000:1.0:1782488178.269796:0:213227:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298645504 00000100:00100000:1.0:1782488178.269816:0:213227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 728 00000100:00100000:1.0:1782488178.269823:0:213227:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4a57b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2360:x1869066298645376:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:1.0:1782488178.269854:0:213227:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4a57b5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2360:x1869066298645376:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 30us (1002us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488178.269860:0:213227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 728 00000100:00100000:1.0:1782488178.269872:0:213227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 729 00000100:00100000:1.0:1782488178.269878:0:213227:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4a57b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2361:x1869066298645504:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:1.0:1782488178.269914:0:213227:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4a57b6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2361:x1869066298645504:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 35us (1031us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488178.269921:0:213227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 729 00000100:00080000:1.0:1782488178.270024:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1782488178.270028:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1782488178.309205:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488179.333920:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488180.355065:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488181.385794:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1782488181.572374:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb453459c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3314:1869066300129792:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1782488181.572469:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488181.572528:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb476c0b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3314:1869066300130304:0@lo:400:kworker.0 00000100:00100000:0.0:1782488181.572553:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488181.572575:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1782488181.572581:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb453458a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300129920:0@lo:400:kworker.0 00000100:00100000:0.0:1782488181.572628:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488181.572651:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb57ff2ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3314:1869066300130432:0@lo:400:kworker.0 00000100:00100000:0.0:1782488181.572673:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488181.572699:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb47e59d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300130048:0@lo:400:kworker.0 00000100:00100000:0.0:1782488181.572725:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488181.572744:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb57ff2c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3316:1869066300130560:0@lo:400:kworker.0 00000100:00100000:0.0:1782488181.572781:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488181.572800:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1782488181.572806:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb47e59e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300130176:0@lo:400:kworker.0 00000100:00100000:0.0:1782488181.572827:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488181.572920:0:190932:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300129792 00000100:00100000:0.0:1782488181.572927:0:190932:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300129792 00000100:00100000:0.0:1782488181.572980:0:190932:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 449 00000100:00100000:0.0:1782488181.572988:0:190932:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb44ed08e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1869066300129792:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1782488181.573021:0:190932:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 30343168 pending 0 free 3806330880 avail 3804233728 00000100:00100000:0.0:1782488181.573079:0:190932:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb44ed08e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1869066300129792:12345-0@lo:13:osp-pre-1-0.0 Request processed in 91us (610us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488181.573086:0:190932:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 449 00000100:00100000:0.0:1782488181.573109:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300130304 00000100:00100000:0.0:1782488181.573112:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300130304 00000100:00100000:0.0:1782488181.573258:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300130432 00000100:00100000:0.0:1782488181.573269:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300130432 00000100:00100000:0.0:1782488181.573291:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300130560 00000100:00100000:0.0:1782488181.573294:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300130560 00000100:00100000:0.0:1782488181.573312:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 942 00000100:00100000:0.0:1782488181.573318:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb44ed0a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3314:x1869066300130304:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488181.573354:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb44ed0a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3314:x1869066300130304:12345-0@lo:400:kworker.0 Request processed in 36us (799us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488181.573360:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 942 00000100:00100000:0.0:1782488181.573373:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 943 00000100:00100000:0.0:1782488181.573379:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb44ed09180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3314:x1869066300130432:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488181.573406:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb44ed09180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3314:x1869066300130432:12345-0@lo:400:kworker.0 Request processed in 27us (732us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488181.573412:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 943 00000100:00100000:0.0:1782488181.573423:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 944 00000100:00100000:0.0:1782488181.573428:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4a41db480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3316:x1869066300130560:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488181.573455:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4a41db480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3316:x1869066300130560:12345-0@lo:400:kworker.0 Request processed in 27us (673us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488181.573460:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 944 00000100:00100000:0.0:1782488181.573476:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300129920 00000100:00100000:0.0:1782488181.573479:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300129920 00000100:00100000:0.0:1782488181.573501:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 575 00000100:00100000:0.0:1782488181.573507:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb545d18850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+19:3314:x1869066300129920:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488181.573602:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb545d18850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+19:3314:x1869066300129920:12345-0@lo:400:kworker.0 Request processed in 94us (972us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488181.573635:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 575 00000100:00100000:0.0:1782488181.573667:0:213225:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300130048 00000100:00100000:0.0:1782488181.573671:0:213225:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300130048 00000100:00100000:0.0:1782488181.573707:0:213225:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300130176 00000100:00100000:0.0:1782488181.573711:0:213225:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300130176 00000100:00100000:0.0:1782488181.573728:0:213225:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 730 00000100:00100000:0.0:1782488181.573733:0:213225:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb549180700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300130048:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488181.573766:0:213225:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb549180700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300130048:12345-0@lo:400:kworker.0 Request processed in 33us (1040us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488181.573773:0:213225:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 730 00000100:00100000:0.0:1782488181.573784:0:213225:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 731 00000100:00100000:0.0:1782488181.573790:0:213225:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4a41db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300130176:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488181.573818:0:213225:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4a41db100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300130176:12345-0@lo:400:kworker.0 Request processed in 28us (990us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488181.573823:0:213225:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 731 00000100:00080000:0.0:1782488181.573844:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1782488181.573847:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1782488181.573873:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb453458a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300129920:0@lo:400:kworker.0 00000100:00100000:0.0:1782488181.573887:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb57ff2ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3314:1869066300130432:0@lo:400:kworker.0 00000100:00100000:0.0:1782488181.573902:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb453459c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3314:1869066300129792:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1782488181.573913:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb476c0b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3314:1869066300130304:0@lo:400:kworker.0 00000100:00100000:0.0:1782488181.573956:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb47e59e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300130176:0@lo:400:kworker.0 00000100:00100000:0.0:1782488181.573965:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb47e59d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300130048:0@lo:400:kworker.0 00000100:00100000:0.0:1782488181.573973:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb57ff2c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3316:1869066300130560:0@lo:400:kworker.0 00000100:00100000:1.0:1782488181.592825:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4a57b5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3317:1869066300130688:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1782488181.592894:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488181.592943:0:198596:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300130688 00000100:00100000:1.0:1782488181.592949:0:198596:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300130688 00000100:00100000:1.0:1782488181.592990:0:198596:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 450 00000100:00100000:1.0:1782488181.592999:0:198596:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4a57b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1869066300130688:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1782488181.593049:0:198596:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 38731776 pending 0 free 3825205248 avail 3823108096 00000100:00100000:1.0:1782488181.593091:0:198596:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4a57b5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3317:x1869066300130688:12345-0@lo:13:osp-pre-0-0.0 Request processed in 92us (196us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488181.593097:0:198596:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 450 00000100:00100000:1.0:1782488181.593141:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4a57b5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3317:1869066300130688:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1782488182.408999:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1782488182.736910:0:228942:0:(debug.c:654:libcfs_debug_mark_buffer()) DEBUG MARKER: oleg221-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8a6ac800b000.ost_server_uuid 50 00000100:00100000:0.0:1782488183.383509:0:3307:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:0.0:1782488183.383549:0:3307:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:0.0:1782488183.383585:0:3307:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:0.0:1782488183.383646:0:3307:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:0.0:1782488183.383676:0:190928:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298646016 00000100:00100000:0.0:1782488183.383685:0:190928:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298646016 00000100:00100000:0.0:1782488183.383779:0:190928:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298645888 00000100:00100000:0.0:1782488183.383782:0:190928:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298645888 00000100:00100000:0.0:1782488183.383806:0:190928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 732 00000100:00100000:0.0:1782488183.383816:0:190928:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4a41db800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2361:x1869066298646016:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:0.0:1782488183.383882:0:190928:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4a41db800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2361:x1869066298646016:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 68us (332us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488183.383889:0:190928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 732 00000100:00100000:0.0:1782488183.383907:0:190928:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 733 00000100:00100000:0.0:1782488183.383913:0:190928:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb47e59d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2361:x1869066298645888:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:0.0:1782488183.383935:0:190928:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb47e59d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2361:x1869066298645888:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 23us (290us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488183.383940:0:190928:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 733 00000100:00080000:0.0:1782488183.383958:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1782488183.383961:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1782488183.384396:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298645760 00000100:00100000:1.0:1782488183.384402:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298645760 00000100:00100000:1.0:1782488183.384444:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 945 00000100:00100000:1.0:1782488183.384451:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4a41dad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2361:x1869066298645760:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:1.0:1782488183.384480:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4a41dad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2361:x1869066298645760:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 31us (990us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488183.384486:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 945 00000100:00100000:1.0:1782488183.384517:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298645632 00000100:00100000:1.0:1782488183.384527:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298645632 00000100:00100000:1.0:1782488183.384551:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 576 00000100:00100000:1.0:1782488183.384556:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f342c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2361:x1869066298645632:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:1.0:1782488183.384578:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f342c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2361:x1869066298645632:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 22us (993us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488183.384584:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 576 00010000:00010000:2.0:1782488183.428725:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488184.452061:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1782488184.670287:0:228990:0:(debug.c:654:libcfs_debug_mark_buffer()) DEBUG MARKER: osc.lustre-OST0000-osc-ffff8a6ac800b000.ost_server_uuid in FULL state after 0 sec 00010000:00010000:2.0:1782488185.482840:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488186.504911:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1782488186.696399:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4a57b5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3314:1869066300130816:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1782488186.696490:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488186.696569:0:190933:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300130816 00000100:00100000:1.0:1782488186.696576:0:190933:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300130816 00000100:00100000:1.0:1782488186.696649:0:190933:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 451 00000100:00100000:1.0:1782488186.696659:0:190933:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4a57b7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1869066300130816:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1782488186.696700:0:190933:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 38731776 pending 0 free 3825205248 avail 3823108096 00000100:00100000:1.0:1782488186.696746:0:190933:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4a57b7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1869066300130816:12345-0@lo:13:osp-pre-0-0.0 Request processed in 89us (256us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488186.696754:0:190933:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 451 00000100:00080000:1.0:1782488186.696777:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1782488186.696812:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4a57b5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3314:1869066300130816:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1782488186.699601:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4a57b5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3315:1869066300130944:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1782488186.699677:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488186.699735:0:190932:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300130944 00000100:00100000:1.0:1782488186.699741:0:190932:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300130944 00000100:00100000:1.0:1782488186.699785:0:190932:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 452 00000100:00100000:1.0:1782488186.699794:0:190932:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4a57b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3315:x1869066300130944:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1782488186.699835:0:190932:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 30343168 pending 0 free 3806330880 avail 3804233728 00000100:00100000:1.0:1782488186.699880:0:190932:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4a57b4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3315:x1869066300130944:12345-0@lo:13:osp-pre-1-0.0 Request processed in 86us (201us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488186.699887:0:190932:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 452 00000100:00080000:1.0:1782488186.699908:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1782488186.699940:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4a57b5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3315:1869066300130944:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1782488186.710581:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb47e59e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300131072:0@lo:400:kworker.0 00000100:00100000:1.0:1782488186.710723:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488186.710774:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1782488186.710781:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb476c0b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3316:1869066300131200:0@lo:400:kworker.0 00000100:00100000:1.0:1782488186.710813:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488186.710847:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb57ff2c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3314:1869066300131328:0@lo:400:kworker.0 00000100:00100000:1.0:1782488186.710870:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488186.710887:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1782488186.710893:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb57ff2ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3314:1869066300131456:0@lo:400:kworker.0 00000100:00100000:1.0:1782488186.710913:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488186.710945:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300131072 00000100:00100000:1.0:1782488186.710953:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300131072 00000100:00100000:1.0:1782488186.711009:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 577 00000100:00100000:1.0:1782488186.711018:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f343450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+19:3316:x1869066300131072:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488186.711075:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f343450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+19:3316:x1869066300131072:12345-0@lo:400:kworker.0 Request processed in 58us (352us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488186.711513:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 577 00000100:00100000:1.0:1782488186.711556:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300131200 00000100:00100000:1.0:1782488186.711560:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300131200 00000100:00100000:1.0:1782488186.711601:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300131328 00000100:00100000:1.0:1782488186.711621:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300131328 00000100:00100000:1.0:1782488186.711640:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300131456 00000100:00100000:1.0:1782488186.711643:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300131456 00000100:00100000:1.0:1782488186.711661:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 946 00000100:00100000:1.0:1782488186.711668:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4a57b5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1869066300131200:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488186.711705:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4a57b5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1869066300131200:12345-0@lo:400:kworker.0 Request processed in 37us (890us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488186.711711:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 946 00000100:00100000:1.0:1782488186.711725:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 947 00000100:00100000:1.0:1782488186.711731:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4a57b4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3314:x1869066300131328:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488186.711762:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4a57b4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3314:x1869066300131328:12345-0@lo:400:kworker.0 Request processed in 32us (891us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488186.711768:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 947 00000100:00100000:1.0:1782488186.711779:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 948 00000100:00100000:1.0:1782488186.711785:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4a57b7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3314:x1869066300131456:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488186.711811:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4a57b7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3314:x1869066300131456:12345-0@lo:400:kworker.0 Request processed in 27us (896us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488186.711816:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 948 00000100:00100000:1.0:1782488186.711861:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb476c0b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3316:1869066300131200:0@lo:400:kworker.0 00000100:00100000:1.0:1782488186.711871:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb47e59e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300131072:0@lo:400:kworker.0 00000100:00100000:1.0:1782488186.711909:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb57ff2ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3314:1869066300131456:0@lo:400:kworker.0 00000100:00100000:1.0:1782488186.711917:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb57ff2c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3314:1869066300131328:0@lo:400:kworker.0 00000001:02000400:0.0:1782488187.200696:0:229039:0:(debug.c:654:libcfs_debug_mark_buffer()) DEBUG MARKER: cleanup: ====================================================== 00010000:00010000:2.0:1782488187.528050:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1782488188.515807:0:3308:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:0.0:1782488188.515856:0:3308:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:0.0:1782488188.515882:0:3308:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:0.0:1782488188.515921:0:3308:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:0.0:1782488188.515950:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298646144 00000100:00100000:0.0:1782488188.515960:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298646144 00000100:00100000:0.0:1782488188.516022:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 578 00000100:00100000:0.0:1782488188.516032:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f343c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2363:x1869066298646144:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:0.0:1782488188.516127:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f343c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2363:x1869066298646144:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 96us (329us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488188.516136:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 578 00000100:00100000:0.0:1782488188.516171:0:213210:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298646400 00000100:00100000:0.0:1782488188.516174:0:213210:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298646400 00000100:00100000:0.0:1782488188.516207:0:213210:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298646528 00000100:00100000:0.0:1782488188.516210:0:213210:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298646528 00000100:00100000:0.0:1782488188.516231:0:213210:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 734 00000100:00100000:0.0:1782488188.516237:0:213210:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57ff2c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2363:x1869066298646400:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:0.0:1782488188.516268:0:213210:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57ff2c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2363:x1869066298646400:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 30us (411us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488188.516274:0:213210:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 734 00000100:00100000:0.0:1782488188.516285:0:213210:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 735 00000100:00100000:0.0:1782488188.516291:0:213210:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb453459c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2363:x1869066298646528:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:0.0:1782488188.516311:0:213210:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb453459c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2363:x1869066298646528:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 21us (390us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488188.516317:0:213210:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 735 00000100:00100000:0.0:1782488188.516332:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298646272 00000100:00100000:0.0:1782488188.516335:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298646272 00000100:00100000:0.0:1782488188.516358:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 949 00000100:00100000:0.0:1782488188.516363:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57ff2ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2363:x1869066298646272:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:0.0:1782488188.516385:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57ff2ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2363:x1869066298646272:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 23us (503us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488188.516391:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 949 00000100:00080000:0.0:1782488188.516499:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1782488188.516510:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1782488188.548269:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488189.578089:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1782488189.891764:0:229087:0:(debug.c:654:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanityn test complete, duration 7520 sec ============== 11:36:27 (1782488187) 00010000:00010000:2.0:1782488190.594782:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488191.619604:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1782488191.671027:0:229135:0:(debug.c:654:libcfs_debug_mark_buffer()) DEBUG MARKER: === sanityn: start cleanup 11:36:30 (1782488190) === 00000100:00100000:1.0:1782488191.810103:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4a57b4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300131584:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1782488191.810188:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488191.810245:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1782488191.810251:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4a57b7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300131712:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1782488191.810271:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488191.810300:0:198596:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300131584 00000100:00100000:1.0:1782488191.810307:0:198596:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300131584 00000100:00100000:1.0:1782488191.810361:0:198596:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300131712 00000100:00100000:1.0:1782488191.810365:0:198596:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300131712 00000100:00100000:1.0:1782488191.810383:0:198596:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 453 00000100:00100000:1.0:1782488191.810391:0:198596:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4a57b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300131584:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1782488191.810440:0:198596:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 30343168 pending 0 free 3806330880 avail 3804233728 00000100:00100000:1.0:1782488191.810487:0:198596:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4a57b5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300131584:12345-0@lo:13:osp-pre-1-0.0 Request processed in 96us (298us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488191.810494:0:198596:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 453 00000100:00100000:1.0:1782488191.810513:0:198596:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 454 00000100:00100000:1.0:1782488191.810520:0:198596:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4a57b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300131712:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1782488191.810538:0:198596:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 38731776 pending 0 free 3825205248 avail 3823108096 00000100:00100000:1.0:1782488191.810565:0:198596:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4a57b4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300131712:12345-0@lo:13:osp-pre-0-0.0 Request processed in 46us (293us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488191.810571:0:198596:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 454 00000100:00080000:1.0:1782488191.810590:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1782488191.810594:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1782488191.810644:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4a57b7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300131712:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1782488191.810655:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4a57b4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300131584:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1782488191.821537:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442933100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300131840:0@lo:400:kworker.0 00000100:00100000:1.0:1782488191.821630:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488191.821692:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1782488191.821699:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442932680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3314:1869066300131968:0@lo:400:kworker.0 00000100:00100000:1.0:1782488191.821730:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488191.821765:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442933480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3316:1869066300132096:0@lo:400:kworker.0 00000100:00100000:1.0:1782488191.821788:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488191.821805:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1782488191.821811:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442933b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3316:1869066300132224:0@lo:400:kworker.0 00000100:00100000:1.0:1782488191.821830:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488191.821892:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300131840 00000100:00100000:1.0:1782488191.821899:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300131840 00000100:00100000:1.0:1782488191.821957:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 579 00000100:00100000:1.0:1782488191.821965:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f342050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+19:3314:x1869066300131840:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488191.822019:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f342050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+19:3314:x1869066300131840:12345-0@lo:400:kworker.0 Request processed in 54us (388us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488191.822026:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 579 00000100:00100000:1.0:1782488191.822062:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300131968 00000100:00100000:1.0:1782488191.822066:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300131968 00000100:00100000:1.0:1782488191.822088:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300132096 00000100:00100000:1.0:1782488191.822090:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300132096 00000100:00100000:1.0:1782488191.822106:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300132224 00000100:00100000:1.0:1782488191.822108:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300132224 00000100:00100000:1.0:1782488191.822127:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 950 00000100:00100000:1.0:1782488191.822132:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4a57b4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3314:x1869066300131968:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488191.822165:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4a57b4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0000_UUID+5:3314:x1869066300131968:12345-0@lo:400:kworker.0 Request processed in 32us (434us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488191.822171:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 950 00000100:00100000:1.0:1782488191.822182:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 951 00000100:00100000:1.0:1782488191.822187:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4a57b7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3316:x1869066300132096:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488191.822228:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4a57b7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3316:x1869066300132096:12345-0@lo:400:kworker.0 Request processed in 40us (439us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488191.822234:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 951 00000100:00100000:1.0:1782488191.822246:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 952 00000100:00100000:1.0:1782488191.822252:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4a57b6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3316:x1869066300132224:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488191.822277:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4a57b6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3316:x1869066300132224:12345-0@lo:400:kworker.0 Request processed in 26us (446us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488191.822283:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 952 00000100:00100000:1.0:1782488191.822325:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442932680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3314:1869066300131968:0@lo:400:kworker.0 00000100:00100000:1.0:1782488191.822335:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442933100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300131840:0@lo:400:kworker.0 00000100:00100000:1.0:1782488191.822366:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442933b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3316:1869066300132224:0@lo:400:kworker.0 00000100:00100000:1.0:1782488191.822376:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442933480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3316:1869066300132096:0@lo:400:kworker.0 00010000:00010000:1.0:1782488192.642732:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1782488193.646747:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:2.0:1782488193.646809:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:2.0:1782488193.646850:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:2.0:1782488193.646907:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:2.0:1782488193.646949:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298646784 00000100:00100000:2.0:1782488193.646957:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298646784 00000100:00100000:2.0:1782488193.647013:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 953 00000100:00100000:2.0:1782488193.647023:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb44fa0df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2360:x1869066298646784:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:2.0:1782488193.647094:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb44fa0df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2360:x1869066298646784:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 72us (243us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488193.647116:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 953 00000100:00100000:2.0:1782488193.647165:0:213211:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298646912 00000100:00100000:2.0:1782488193.647169:0:213211:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298646912 00000100:00100000:2.0:1782488193.647218:0:213211:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298647040 00000100:00100000:2.0:1782488193.647221:0:213211:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298647040 00000100:00100000:2.0:1782488193.647240:0:213211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 736 00000100:00100000:2.0:1782488193.647260:0:213211:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb44fa0ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2360:x1869066298646912:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:2.0:1782488193.647286:0:213211:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb44fa0ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2360:x1869066298646912:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 40us (477us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488193.647292:0:213211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 736 00000100:00100000:2.0:1782488193.647303:0:213211:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 737 00000100:00100000:2.0:1782488193.647309:0:213211:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb497040700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2360:x1869066298647040:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:2.0:1782488193.647341:0:213211:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb497040700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2360:x1869066298647040:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 33us (435us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488193.647347:0:213211:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 737 00000100:00080000:2.0:1782488193.647364:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1782488193.647368:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1782488193.647968:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298646656 00000100:00100000:0.0:1782488193.647973:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298646656 00000100:00100000:0.0:1782488193.648008:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 580 00000100:00100000:0.0:1782488193.648015:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f341850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2360:x1869066298646656:12345-192.168.202.21@tcp:400:kworker.0 00000100:00100000:0.0:1782488193.648046:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f341850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2360:x1869066298646656:12345-192.168.202.21@tcp:400:kworker.0 Request processed in 33us (1335us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488193.648052:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 580 00010000:00010000:3.0:1782488193.677128:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488194.692238:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1782488195.365174:0:3307:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:0.0:1782488195.367790:0:3307:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:0.0:1782488195.373251:0:3307:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:2.0:1782488195.379047:0:192469:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298647424 00000100:00100000:2.0:1782488195.379062:0:192469:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298647424 00000100:00100000:2.0:1782488195.379151:0:192469:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298647168 00000100:00100000:2.0:1782488195.379155:0:192469:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298647168 00000100:00100000:2.0:1782488195.379185:0:192469:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 1219 00000100:00100000:2.0:1782488195.379194:0:192469:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb453458a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:60894528-77cf-40cf-92ce-b54cc4bbae30+11:2362:x1869066298647424:12345-192.168.202.21@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1782488195.379213:0:192469:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1782488195.379234:0:192469:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff9cb546d35800/0x66475b4e8b7973fa lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 6 type: PLN flags: 0x44801000000000 nid: 192.168.202.21@tcp remote: 0x43f633357a441975 expref: 11 pid: 190290 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782488195.379273:0:192469:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1782488195.379350:0:192469:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb453458a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:60894528-77cf-40cf-92ce-b54cc4bbae30+10:2362:x1869066298647424:12345-192.168.202.21@tcp:103:ldlm_bl.0 Request processed in 156us (14186us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488195.379358:0:192469:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 1219 00000100:00100000:2.0:1782488195.379549:0:192469:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 1220 00000100:00100000:2.0:1782488195.379558:0:192469:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4cf558000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:60894528-77cf-40cf-92ce-b54cc4bbae30+9:2362:x1869066298647168:12345-192.168.202.21@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1782488195.379567:0:192469:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1782488195.379577:0:192469:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff9cb54834d000/0x66475b4e8b797391 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x44801000000000 nid: 192.168.202.21@tcp remote: 0x43f633357a441913 expref: 9 pid: 190292 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782488195.379586:0:192469:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1782488195.379636:0:192469:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4cf558000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:60894528-77cf-40cf-92ce-b54cc4bbae30+8:2362:x1869066298647168:12345-192.168.202.21@tcp:103:ldlm_bl.0 Request processed in 50us (11826us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488195.379644:0:192469:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 1220 00000100:00100000:0.0:1782488195.382200:0:3307:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:0.0:1782488195.382246:0:213227:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298647680 00000100:00100000:0.0:1782488195.382257:0:213227:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298647680 00000100:00100000:0.0:1782488195.382471:0:213227:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 20823 00000100:00100000:0.0:1782488195.382482:0:213227:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579574a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:5c4ca15f-f34e-4111-b105-de67ffe3ec92+6:2359:x1869066298647680:12345-192.168.202.21@tcp:9:umount.0 00000020:00000080:0.0:1782488195.382540:0:213227:0:(genops.c:1489:class_disconnect()) disconnect: cookie 0x66475b4e8b8c8a63 00000020:00010000:0.0:1782488195.382549:0:213227:0:(genops.c:1942:obd_stale_export_put()) Put export ffff9cb551575000: total 0 00000100:00080000:0.0:1782488195.382553:0:213227:0:(import.c:65:import_set_state_nolock()) ffff9cb551571800 : changing import state from FULL to CLOSED 00000001:00000004:0.0:1782488195.382591:0:213227:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0001_UUID: mount_count is 2, last_transno is 8589941508 00000100:00100000:0.0:1782488195.382931:0:213227:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579574a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:5c4ca15f-f34e-4111-b105-de67ffe3ec92+4:2359:x1869066298647680:12345-192.168.202.21@tcp:9:umount.0 Request processed in 449us (738us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488195.382940:0:213227:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 20823 00000100:00080000:0.0:1782488195.382964:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000020:00010000:0.0:1782488195.382978:0:190285:0:(genops.c:1928:obd_stale_export_get()) Get export ffff9cb551575000: total 0 00000100:00100000:2.0:1782488195.383122:0:192528:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298647296 00000100:00100000:2.0:1782488195.383127:0:192528:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298647296 00000100:00100000:2.0:1782488195.383167:0:192528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 1221 00000100:00100000:2.0:1782488195.383174:0:192528:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579577100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:60894528-77cf-40cf-92ce-b54cc4bbae30+8:2363:x1869066298647296:12345-192.168.202.21@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1782488195.383185:0:192528:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1782488195.383203:0:192528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff9cb576a0d400/0x66475b4e8b797398 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 4 type: PLN flags: 0x44801000000000 nid: 192.168.202.21@tcp remote: 0x43f633357a44193d expref: 8 pid: 190291 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782488195.383219:0:192528:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1782488195.383241:0:192528:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579577100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:60894528-77cf-40cf-92ce-b54cc4bbae30+7:2363:x1869066298647296:12345-192.168.202.21@tcp:103:ldlm_bl.0 Request processed in 67us (10000us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488195.383257:0:192528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 1221 00010000:00010000:0.0:1782488195.384109:0:190285:0:(ldlm_lock.c:2652:ldlm_export_cancel_blocked_locks()) Export ffff9cb551575000, canceled 0 locks, left on hash table 1. 00010000:00010000:0.0:1782488195.384187:0:190285:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000cc722909/0x66475b4e8b8c9d4f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1d4b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 4096->1048575) gid 0 flags: 0x44801000020000 nid: 192.168.202.21@tcp remote: 0x43f633357a48a77a expref: 3 pid: 213223 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488195.384211:0:190285:0:(ldlm_lock.c:2680:ldlm_export_cancel_locks()) Export ffff9cb551575000, canceled 1 locks, left on hash table 0. 00010000:00010000:0.0:1782488195.384223:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488195.384226:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488195.384227:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1782488195.392310:0:3309:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:0.0:1782488195.392356:0:192482:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298648064 00000100:00100000:0.0:1782488195.392376:0:192482:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298648064 00000100:00100000:0.0:1782488195.392439:0:192482:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 20824 00000100:00100000:0.0:1782488195.392449:0:192482:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579574380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5c4ca15f-f34e-4111-b105-de67ffe3ec92+7:2359:x1869066298648064:12345-192.168.202.21@tcp:9:umount.0 00000020:00000080:0.0:1782488195.392483:0:192482:0:(genops.c:1489:class_disconnect()) disconnect: cookie 0x66475b4e8b8c8a55 00000020:00010000:0.0:1782488195.392490:0:192482:0:(genops.c:1942:obd_stale_export_put()) Put export ffff9cb551574000: total 0 00000100:00080000:0.0:1782488195.392494:0:192482:0:(import.c:65:import_set_state_nolock()) ffff9cb551572800 : changing import state from FULL to CLOSED 00000001:00000004:0.0:1782488195.392530:0:192482:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0000_UUID: mount_count is 2, last_transno is 8589942216 00000100:00100000:0.0:1782488195.392799:0:192482:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579574380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:5c4ca15f-f34e-4111-b105-de67ffe3ec92+5:2359:x1869066298648064:12345-192.168.202.21@tcp:9:umount.0 Request processed in 350us (497us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488195.392805:0:192482:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 20824 00000100:00080000:0.0:1782488195.392825:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000020:00010000:0.0:1782488195.392835:0:213229:0:(genops.c:1928:obd_stale_export_get()) Get export ffff9cb551574000: total 0 00010000:00010000:0.0:1782488195.392853:0:213229:0:(ldlm_lock.c:2652:ldlm_export_cancel_blocked_locks()) Export ffff9cb551574000, canceled 0 locks, left on hash table 2. 00010000:00010000:0.0:1782488195.392950:0:213229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004deed3a7/0x66475b4e8b8c9d5d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1d73:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 1048576->18446744073709551615) gid 0 flags: 0x44801000020000 nid: 192.168.202.21@tcp remote: 0x43f633357a48a788 expref: 4 pid: 213224 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488195.393000:0:213229:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000779e1df2/0x66475b4e8b8c9b9d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1d72:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801000020000 nid: 192.168.202.21@tcp remote: 0x43f633357a48a6d9 expref: 3 pid: 213221 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488195.393050:0:213229:0:(ldlm_lock.c:2680:ldlm_export_cancel_locks()) Export ffff9cb551574000, canceled 2 locks, left on hash table 0. 00010000:00010000:0.0:1782488195.393062:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488195.393064:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488195.393066:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1782488195.409203:0:3308:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:2.0:1782488195.409255:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298649344 00000100:00100000:2.0:1782488195.409267:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298649344 00000100:00100000:2.0:1782488195.409343:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 14002 00000100:00100000:2.0:1782488195.409355:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb548844e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5c4ca15f-f34e-4111-b105-de67ffe3ec92+16:2359:x1869066298649344:12345-192.168.202.21@tcp:39:umount.0 00000020:00000080:2.0:1782488195.409411:0:196437:0:(genops.c:1489:class_disconnect()) disconnect: cookie 0x66475b4e8b7973a6 00000020:00010000:2.0:1782488195.409420:0:196437:0:(genops.c:1942:obd_stale_export_put()) Put export ffff9cb551430000: total 0 00000100:00080000:2.0:1782488195.409425:0:196437:0:(import.c:65:import_set_state_nolock()) ffff9cb56fda2800 : changing import state from FULL to CLOSED 00000001:00000004:2.0:1782488195.409447:0:196437:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-MDT0000_UUID: mount_count is 2, last_transno is 8589967276 00000100:00100000:2.0:1782488195.409940:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb548844e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:5c4ca15f-f34e-4111-b105-de67ffe3ec92+15:2359:x1869066298649344:12345-192.168.202.21@tcp:39:umount.0 Request processed in 586us (745us total) trans 8589967277 rc 0/0 00000100:00100000:2.0:1782488195.409949:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 14002 00000020:00010000:2.0:1782488195.409999:0:191970:0:(genops.c:1928:obd_stale_export_get()) Get export ffff9cb551430000: total 0 00010000:00010000:2.0:1782488195.410002:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488195.410052:0:191970:0:(ldlm_lock.c:2652:ldlm_export_cancel_blocked_locks()) Export ffff9cb551430000, canceled 0 locks, left on hash table 9. 00010000:00010000:2.0:1782488195.410084:0:191970:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd1:0x1535:0x0].0x0 (ffff9cb550ee4e00) 00010000:00010000:2.0:1782488195.410088:0:191970:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1782488195.410095:0:191970:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cb488319400/0x66475b4e8b8c9840 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x1535:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.21@tcp remote: 0x43f633357a48a5ac expref: 14 pid: 196437 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782488195.410144:0:191970:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd1:0x1530:0x0].0x0 (ffff9cb495693a00) 00010000:00010000:2.0:1782488195.410148:0:191970:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1782488195.410152:0:191970:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cb4bd7d2800/0x66475b4e8b8c9695 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x1530:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.21@tcp remote: 0x43f633357a48a519 expref: 13 pid: 213935 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782488195.410170:0:191970:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff9cb579f0f300) 00010000:00010000:2.0:1782488195.410186:0:191970:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1782488195.410191:0:191970:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cb44f586e00/0x66475b4e8b8c9de9 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.21@tcp remote: 0x43f633357a48a806 expref: 12 pid: 213218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782488195.410215:0:191970:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd1:0x153b:0x0].0x0 (ffff9cb4b4a53a00) 00010000:00010000:2.0:1782488195.410217:0:191970:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1782488195.410222:0:191970:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cb45d8eb000/0x66475b4e8b8c9d41 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x153b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.21@tcp remote: 0x43f633357a48a773 expref: 11 pid: 196437 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782488195.410236:0:191970:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd1:0x1534:0x0].0x0 (ffff9cb550ee4000) 00010000:00010000:2.0:1782488195.410239:0:191970:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1782488195.410242:0:191970:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cb488319800/0x66475b4e8b8c9824 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x1534:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.21@tcp remote: 0x43f633357a48a5a5 expref: 10 pid: 190296 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782488195.410256:0:191970:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd1:0x1539:0x0].0x0 (ffff9cb4b4a53200) 00010000:00010000:2.0:1782488195.410258:0:191970:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1782488195.410261:0:191970:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cb488319e00/0x66475b4e8b8c9ae0 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x1539:0x0].0x0 bits 0x1a/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.21@tcp remote: 0x43f633357a48a693 expref: 9 pid: 213935 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782488195.410275:0:191970:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd1:0x153a:0x0].0x0 (ffff9cb4595ddc00) 00010000:00010000:2.0:1782488195.410277:0:191970:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1782488195.410281:0:191970:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cb471760600/0x66475b4e8b8c9b8f lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x153a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.21@tcp remote: 0x43f633357a48a6d2 expref: 8 pid: 196437 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782488195.410311:0:191970:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd2:0xb22:0x0].0x0 (ffff9cb579f0ff00) 00010000:00010000:2.0:1782488195.410314:0:191970:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1782488195.410318:0:191970:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cb44f586000/0x66475b4e8b8c9ddb lrc: 0/0,0 mode: --/CR res: [0x200000bd2:0xb22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.21@tcp remote: 0x43f633357a48a7ff expref: 7 pid: 190296 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782488195.410349:0:191970:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000bd1:0x1533:0x0].0x0 (ffff9cb4595dd600) 00010000:00010000:2.0:1782488195.410351:0:191970:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1782488195.410355:0:191970:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff9cb4bd7d2000/0x66475b4e8b8c9744 lrc: 0/0,0 mode: --/PR res: [0x200000bd1:0x1533:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.21@tcp remote: 0x43f633357a48a551 expref: 6 pid: 213218 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782488195.410370:0:191970:0:(ldlm_lock.c:2680:ldlm_export_cancel_locks()) Export ffff9cb551430000, canceled 9 locks, left on hash table 0. 00010000:00010000:2.0:1782488195.410382:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488195.410384:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488195.410385:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1782488195.438643:0:3307:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.21@tcp (source: 12345-192.168.202.21@tcp) 00000100:00100000:2.0:1782488195.438960:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066298649600 00000100:00100000:2.0:1782488195.438972:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066298649600 00000100:00100000:2.0:1782488195.439038:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.21@tcp, seq: 581 00000100:00100000:2.0:1782488195.439049:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f343050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:60894528-77cf-40cf-92ce-b54cc4bbae30+7:2359:x1869066298649600:12345-192.168.202.21@tcp:251:umount.0 00000020:00000080:2.0:1782488195.439099:0:190292:0:(genops.c:1489:class_disconnect()) disconnect: cookie 0x66475b4e8b79734b 00000020:00010000:2.0:1782488195.439107:0:190292:0:(genops.c:1942:obd_stale_export_put()) Put export ffff9cb4498f7000: total 0 00000100:00080000:2.0:1782488195.439112:0:190292:0:(import.c:65:import_set_state_nolock()) ffff9cb4498f1000 : changing import state from FULL to CLOSED 00000100:00100000:2.0:1782488195.439194:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f343050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:60894528-77cf-40cf-92ce-b54cc4bbae30+3:2359:x1869066298649600:12345-192.168.202.21@tcp:251:umount.0 Request processed in 146us (560us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488195.439202:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.21@tcp, seq: 581 00000020:00010000:2.0:1782488195.439269:0:190286:0:(genops.c:1928:obd_stale_export_get()) Get export ffff9cb4498f7000: total 0 00010000:00010000:2.0:1782488195.439316:0:190286:0:(ldlm_lock.c:2652:ldlm_export_cancel_blocked_locks()) Export ffff9cb4498f7000, canceled 0 locks, left on hash table 1. 00010000:00010000:2.0:1782488195.439345:0:190286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff9cb546d35000/0x66475b4e8b797352 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x44801000000000 nid: 192.168.202.21@tcp remote: 0x43f633357a4418aa expref: 2 pid: 190290 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1782488195.439360:0:190286:0:(ldlm_lock.c:2680:ldlm_export_cancel_locks()) Export ffff9cb4498f7000, canceled 1 locks, left on hash table 0. 00000020:00000080:2.0:1782488195.439376:0:190286:0:(genops.c:906:class_export_put()) final put ffff9cb4498f7000/60894528-77cf-40cf-92ce-b54cc4bbae30 00010000:00010000:2.0:1782488195.439401:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488195.439403:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488195.439404:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:0.0:1782488195.439584:0:223038:0:(genops.c:865:class_export_destroy()) destroying export ffff9cb4498f7000/60894528-77cf-40cf-92ce-b54cc4bbae30 for MGS 00000020:00000080:0.0:1782488195.439637:0:223038:0:(genops.c:1204:obd_zombie_import_free()) destroying import ffff9cb4498f1000 for MGS 00010000:00010000:1.0:1782488195.714502:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488196.737700:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1782488196.931460:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4a57b6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3314:1869066300132352:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1782488196.931530:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488196.931584:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1782488196.931591:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb47e59e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3314:1869066300132480:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1782488196.931630:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488196.931667:0:198596:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300132352 00000100:00100000:1.0:1782488196.931673:0:198596:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300132352 00000100:00100000:1.0:1782488196.931720:0:198596:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300132480 00000100:00100000:1.0:1782488196.931722:0:198596:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300132480 00000100:00100000:1.0:1782488196.931738:0:198596:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 455 00000100:00100000:1.0:1782488196.931745:0:198596:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb476c0b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1869066300132352:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1782488196.931791:0:198596:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 25231360 pending 0 free 3806330880 avail 3804233728 00000100:00100000:1.0:1782488196.931837:0:198596:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb476c0b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1869066300132352:12345-0@lo:13:osp-pre-1-0.0 Request processed in 91us (307us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488196.931844:0:198596:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 455 00000100:00100000:1.0:1782488196.931863:0:198596:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 456 00000100:00100000:1.0:1782488196.931869:0:198596:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb452582a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1869066300132480:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1782488196.932530:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4433fa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300132608:0@lo:400:kworker.0 00000100:00100000:2.0:1782488196.932571:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488196.932600:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1782488196.932622:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4433f8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3316:1869066300132736:0@lo:400:kworker.0 00000100:00100000:2.0:1782488196.932664:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488196.932698:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300132608 00000100:00100000:2.0:1782488196.932703:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300132608 00000100:00100000:2.0:1782488196.932734:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 582 00000100:00100000:2.0:1782488196.932741:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f342850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+19:3316:x1869066300132608:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488196.932776:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f342850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+19:3316:x1869066300132608:12345-0@lo:400:kworker.0 Request processed in 35us (203us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488196.932783:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 582 00000100:00100000:2.0:1782488196.932810:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300132736 00000100:00100000:2.0:1782488196.932813:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300132736 00000100:00100000:2.0:1782488196.932837:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 954 00000100:00100000:2.0:1782488196.932843:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1869066300132736:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488196.932871:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3316:x1869066300132736:12345-0@lo:400:kworker.0 Request processed in 28us (206us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488196.932877:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 954 00000100:00100000:2.0:1782488196.932914:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb48eed5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3315:1869066300132992:0@lo:400:kworker.0 00000100:00100000:2.0:1782488196.932939:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488196.932955:0:3315:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1782488196.932960:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4433f9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3315:1869066300132864:0@lo:400:kworker.0 00000100:00100000:2.0:1782488196.932992:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488196.933022:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300132992 00000100:00100000:2.0:1782488196.933025:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300132992 00000100:00100000:2.0:1782488196.933046:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300132864 00000100:00100000:2.0:1782488196.933048:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300132864 00000100:00100000:2.0:1782488196.933064:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 955 00000100:00100000:2.0:1782488196.933071:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3315:x1869066300132992:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488196.933103:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0001_UUID+5:3315:x1869066300132992:12345-0@lo:400:kworker.0 Request processed in 33us (162us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488196.933108:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 955 00000100:00100000:2.0:1782488196.933122:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 956 00000100:00100000:2.0:1782488196.933128:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3315:x1869066300132864:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488196.933152:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3315:x1869066300132864:12345-0@lo:400:kworker.0 Request processed in 25us (160us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488196.933158:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 956 00000100:00100000:2.0:1782488196.933201:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4433f9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3315:1869066300132864:0@lo:400:kworker.0 00000100:00100000:2.0:1782488196.933212:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb48eed5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3315:1869066300132992:0@lo:400:kworker.0 00000100:00100000:2.0:1782488196.933240:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4433f8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3316:1869066300132736:0@lo:400:kworker.0 00000100:00100000:2.0:1782488196.933248:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4433fa680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300132608:0@lo:400:kworker.0 00002000:00000024:1.0:1782488196.935403:0:198596:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 25100288 pending 0 free 3825205248 avail 3823108096 00000100:00100000:1.0:1782488196.935476:0:198596:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb452582a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3314:x1869066300132480:12345-0@lo:13:osp-pre-0-0.0 Request processed in 3604us (3841us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488196.935486:0:198596:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 456 00000100:00080000:1.0:1782488196.935527:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1782488196.935533:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1782488196.935573:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb47e59e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3314:1869066300132480:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1782488196.935590:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4a57b6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3314:1869066300132352:0@lo:13:osp-pre-1-0.0 00000001:00100000:2.0:1782488197.084100:0:5825:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-MDT0000: committing for initial connect of 5c4ca15f-f34e-4111-b105-de67ffe3ec92 00000001:00080000:2.0:1782488197.084122:0:5825:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 8589967277 is committed 00010000:00010000:2.0:1782488197.765588:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00100000:2.0:1782488198.118982:0:7511:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of 5c4ca15f-f34e-4111-b105-de67ffe3ec92 00000020:00000080:2.0:1782488198.118993:0:7511:0:(genops.c:906:class_export_put()) final put ffff9cb551575000/5c4ca15f-f34e-4111-b105-de67ffe3ec92 00000020:00000080:1.0:1782488198.119110:0:196445:0:(genops.c:865:class_export_destroy()) destroying export ffff9cb551575000/5c4ca15f-f34e-4111-b105-de67ffe3ec92 for lustre-OST0001 00000020:00000080:1.0:1782488198.119175:0:196445:0:(genops.c:1204:obd_zombie_import_free()) destroying import ffff9cb551571800 for lustre-OST0001 00000001:00100000:1.0:1782488198.571248:0:6622:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of 5c4ca15f-f34e-4111-b105-de67ffe3ec92 00000020:00000080:1.0:1782488198.571258:0:6622:0:(genops.c:906:class_export_put()) final put ffff9cb551574000/5c4ca15f-f34e-4111-b105-de67ffe3ec92 00000020:00000080:1.0:1782488198.571297:0:223038:0:(genops.c:865:class_export_destroy()) destroying export ffff9cb551574000/5c4ca15f-f34e-4111-b105-de67ffe3ec92 for lustre-OST0000 00000020:00000080:1.0:1782488198.571359:0:223038:0:(genops.c:1204:obd_zombie_import_free()) destroying import ffff9cb551572800 for lustre-OST0000 00010000:00010000:2.0:1782488198.791024:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488199.819083:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1782488200.083399:0:229236:0:(debug.c:654:libcfs_debug_mark_buffer()) DEBUG MARKER: === sanityn: finish cleanup 11:36:38 (1782488198) === 00010000:00010000:2.0:1782488200.834158:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488201.858405:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1782488202.049791:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452582680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300133120:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1782488202.049874:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488202.049929:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1782488202.049936:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452583800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300133248:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1782488202.049962:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488202.050011:0:198596:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300133120 00000100:00100000:1.0:1782488202.050019:0:198596:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300133120 00000100:00100000:1.0:1782488202.050067:0:198596:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300133248 00000100:00100000:1.0:1782488202.050071:0:198596:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300133248 00000100:00100000:1.0:1782488202.050090:0:198596:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 457 00000100:00100000:1.0:1782488202.050099:0:198596:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4a57b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300133120:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1782488202.050133:0:198596:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 25231360 pending 0 free 3806330880 avail 3804233728 00000100:00100000:1.0:1782488202.050175:0:198596:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4a57b6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300133120:12345-0@lo:13:osp-pre-1-0.0 Request processed in 76us (300us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488202.050259:0:198596:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 457 00000100:00100000:1.0:1782488202.050279:0:198596:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 458 00000100:00100000:1.0:1782488202.050286:0:198596:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb47e59e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300133248:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1782488202.050305:0:198596:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 25100288 pending 0 free 3825205248 avail 3823108096 00000100:00100000:1.0:1782488202.050334:0:198596:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb47e59e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3316:x1869066300133248:12345-0@lo:13:osp-pre-0-0.0 Request processed in 49us (371us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488202.050340:0:198596:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 458 00000100:00080000:1.0:1782488202.050359:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1782488202.050362:0:190293:0:(pinger.c:502:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1782488202.050653:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452583800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300133248:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1782488202.050668:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452582680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3316:1869066300133120:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1782488202.053959:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452582680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300133376:0@lo:400:kworker.0 00000100:00100000:1.0:1782488202.054035:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488202.054080:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1782488202.054087:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452583800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3314:1869066300133504:0@lo:400:kworker.0 00000100:00100000:1.0:1782488202.054118:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488202.054152:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4b47c6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3316:1869066300133632:0@lo:400:kworker.0 00000100:00100000:1.0:1782488202.054277:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488202.054297:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1782488202.054302:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb5589e6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3316:1869066300133760:0@lo:400:kworker.0 00000100:00100000:1.0:1782488202.054322:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488202.054349:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300133376 00000100:00100000:1.0:1782488202.054354:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300133376 00000100:00100000:1.0:1782488202.054390:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 583 00000100:00100000:1.0:1782488202.054399:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f340050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+19:3314:x1869066300133376:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488202.054441:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f340050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+19:3314:x1869066300133376:12345-0@lo:400:kworker.0 Request processed in 43us (407us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488202.054448:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 583 00000100:00100000:1.0:1782488202.054479:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300133504 00000100:00100000:1.0:1782488202.054482:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300133504 00000100:00100000:1.0:1782488202.054511:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300133632 00000100:00100000:1.0:1782488202.054514:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300133632 00000100:00100000:1.0:1782488202.054535:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300133760 00000100:00100000:1.0:1782488202.054538:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300133760 00000100:00100000:1.0:1782488202.054560:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 957 00000100:00100000:1.0:1782488202.054567:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb459637b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3314:x1869066300133504:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488202.054602:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb459637b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3314:x1869066300133504:12345-0@lo:400:kworker.0 Request processed in 35us (482us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488202.054627:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 957 00000100:00100000:1.0:1782488202.054640:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 958 00000100:00100000:1.0:1782488202.054646:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45034aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3316:x1869066300133632:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488202.054673:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45034aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0000_UUID+5:3316:x1869066300133632:12345-0@lo:400:kworker.0 Request processed in 28us (396us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488202.054678:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 958 00000100:00100000:1.0:1782488202.054696:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 959 00000100:00100000:1.0:1782488202.054702:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb450349180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3316:x1869066300133760:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488202.054726:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb450349180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3316:x1869066300133760:12345-0@lo:400:kworker.0 Request processed in 24us (403us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488202.054731:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 959 00000100:00100000:1.0:1782488202.054784:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb5589e6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3316:1869066300133760:0@lo:400:kworker.0 00000100:00100000:1.0:1782488202.054798:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4b47c6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3316:1869066300133632:0@lo:400:kworker.0 00000100:00100000:1.0:1782488202.054822:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452583800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3314:1869066300133504:0@lo:400:kworker.0 00000100:00100000:1.0:1782488202.054830:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452582680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300133376:0@lo:400:kworker.0 00010000:00010000:2.0:1782488202.891748:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:01000004:2.0:1782488203.274190:0:229334:0:(tgt_mount.c:2192:server_umount_begin()) umount -f 00000020:01000004:0.0:1782488203.337819:0:229334:0:(tgt_mount.c:2062:server_put_super()) server put_super lustre-MDT0000 00010000:00010000:0.0:1782488203.337884:0:229334:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.202.121@tcp lock: ffff9cb57fc70a00/0x66475b4e8b79708f lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x11000000000000 nid: local remote: 0x66475b4e8b797096 expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.337900:0:229334:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.121@tcp lock: ffff9cb57fc70a00/0x66475b4e8b79708f lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1000000000000 nid: local remote: 0x66475b4e8b797096 expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.337908:0:229334:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.121@tcp lock: ffff9cb57fc70a00/0x66475b4e8b79708f lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x11000400000000 nid: local remote: 0x66475b4e8b797096 expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.337915:0:229334:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.202.121@tcp lock: ffff9cb57fc70a00/0x66475b4e8b79708f lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x11000400000000 nid: local remote: 0x66475b4e8b797096 expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.337978:0:229334:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 10000000:01000000:0.0:1782488203.337985:0:229334:0:(mgc_request.c:547:config_log_end()) end config log lustre-client (0) 00000100:00080000:0.0:1782488203.338008:0:229334:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import lustre-MDT0000_UUID INVALID 00000100:00100000:0.0:1782488203.338014:0:229334:0:(client.c:3111:ptlrpc_free_committed()) lustre-MDT0000-lwp-MDT0000: committing for last_committed 0 gen 3 00000100:00080000:0.0:1782488203.338019:0:229334:0:(pinger.c:400:ptlrpc_pinger_del_import()) removing pingable import lustre-MDT0000-lwp-MDT0000_UUID->lustre-MDT0000_UUID 00000100:00080000:0.0:1782488203.338047:0:229334:0:(import.c:65:import_set_state_nolock()) ffff9cb5457d7800 lustre-MDT0000_UUID: changing import state from FULL to CONNECTING 00010000:00010000:0.0:1782488203.338091:0:190285:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1782488203.338110:0:190285:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.202.121@tcp lock: ffff9cb57fc70a00/0x66475b4e8b79708f lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1000400000000 nid: local remote: 0x66475b4e8b797096 expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.338117:0:190285:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cb57fc70a00 already unused, calling callback (ffffffffc1d8bc20) 10000000:00010000:0.0:1782488203.338121:0:190285:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.202.121@tcp lock: ffff9cb57fc70a00/0x66475b4e8b79708f lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1000400000000 nid: local remote: 0x66475b4e8b797096 expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.338130:0:190285:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.121@tcp lock: ffff9cb57fc70a00/0x66475b4e8b79708f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1008400000000 nid: local remote: 0x66475b4e8b797096 expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1782488203.338137:0:190285:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.121@tcp lock: ffff9cb57fc70a00/0x66475b4e8b79708f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1009400000000 nid: local remote: 0x66475b4e8b797096 expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1782488203.338143:0:190285:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 10000000:01000000:0.0:1782488203.338146:0:190285:0:(mgc_request.c:944:mgc_blocking_ast()) log lustre-client: stopping, won't requeue 10000000:01000000:0.0:1782488203.338149:0:190285:0:(mgc_request.c:130:config_log_put()) dropping config log lustre-client 00010000:00010000:0.0:1782488203.338172:0:190285:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.202.121@tcp lock: ffff9cb57fc70a00/0x66475b4e8b79708f lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x5809400000020 nid: local remote: 0x66475b4e8b797096 expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.338178:0:190285:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:2.0:1782488203.341991:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579577480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0000_UUID:3313:1869066300133888:0@lo:39:umount.0 00000100:00100000:2.0:1782488203.342113:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1782488203.342216:0:190285:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.202.121@tcp lock: ffff9cb57fc70a00/0x66475b4e8b79708f lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x5809400000020 nid: local remote: 0x66475b4e8b797096 expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.342227:0:190285:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.121@tcp lock: ffff9cb57fc70a00/0x66475b4e8b79708f lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x5809400000020 nid: local remote: 0x66475b4e8b797096 expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.342244:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488203.342246:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488203.342248:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1782488203.342339:0:3315:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1782488203.342355:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579574000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300134016:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1782488203.342419:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488203.342480:0:192469:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300134016 00000100:00100000:0.0:1782488203.342487:0:192469:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300134016 00000100:00100000:0.0:1782488203.342539:0:192469:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1222 00000100:00100000:0.0:1782488203.342549:0:192469:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579577800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:6385baa4-edbe-4eae-af92-80250729af95+19:3315:x1869066300134016:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1782488203.342564:0:192469:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1782488203.342582:0:192469:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff9cb55919dc00/0x66475b4e8b797096 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x44801000000000 nid: 0@lo remote: 0x66475b4e8b79708f expref: 19 pid: 190291 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.342593:0:192469:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1782488203.342644:0:192469:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579577800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:6385baa4-edbe-4eae-af92-80250729af95+18:3315:x1869066300134016:12345-0@lo:103:ldlm_bl.0 Request processed in 97us (224us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488203.342652:0:192469:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1222 00000100:00100000:0.0:1782488203.342695:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579574000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300134016:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1782488203.348683:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300133888 00000100:00100000:0.0:1782488203.348699:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300133888 00000100:00100000:0.0:1782488203.348758:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14003 00000100:00100000:0.0:1782488203.348768:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+5:3313:x1869066300133888:12345-0@lo:39:umount.0 00000020:00000080:0.0:1782488203.348818:0:213218:0:(genops.c:1489:class_disconnect()) disconnect: cookie 0x66475b4e8b7970c0 00000100:00080000:0.0:1782488203.348829:0:213218:0:(import.c:65:import_set_state_nolock()) ffff9cb44108d000 : changing import state from FULL to CLOSED 00000100:00100000:0.0:1782488203.348929:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0000_UUID+1:3313:x1869066300133888:12345-0@lo:39:umount.0 Request processed in 162us (6816us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488203.348937:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14003 00000020:00000080:0.0:1782488203.348948:0:213218:0:(genops.c:906:class_export_put()) final put ffff9cb560b82800/lustre-MDT0000-lwp-MDT0000_UUID 00010000:00010000:0.0:1782488203.348972:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488203.349003:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb5457d7800 lustre-MDT0000_UUID: changing import state from CONNECTING to CLOSED 00000100:00100000:0.0:1782488203.349014:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579577480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0000_UUID:3313:1869066300133888:0@lo:39:umount.0 00000100:00080000:0.0:1782488203.349056:0:229334:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import lustre-MDT0000_UUID INVALID 00000100:00100000:0.0:1782488203.349059:0:229334:0:(client.c:3111:ptlrpc_free_committed()) lustre-MDT0000-lwp-MDT0000: committing for last_committed 0 gen 4 00000100:00100000:0.0:1782488203.349064:0:229334:0:(import.c:323:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 00010000:00010000:0.0:1782488203.349903:0:229334:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.202.121@tcp lock: ffff9cb57fc70600/0x66475b4e8b796ff5 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x11000000000000 nid: local remote: 0x66475b4e8b796ffc expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.349918:0:229334:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.121@tcp lock: ffff9cb57fc70600/0x66475b4e8b796ff5 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1000000000000 nid: local remote: 0x66475b4e8b796ffc expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.349925:0:229334:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.121@tcp lock: ffff9cb57fc70600/0x66475b4e8b796ff5 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x11000400000000 nid: local remote: 0x66475b4e8b796ffc expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.349932:0:229334:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.202.121@tcp lock: ffff9cb57fc70600/0x66475b4e8b796ff5 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x11000400000000 nid: local remote: 0x66475b4e8b796ffc expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.349952:0:229334:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1782488203.349958:0:229334:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.202.121@tcp lock: ffff9cb454f50e00/0x66475b4e8b79729c lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x66475b4e8b7972a3 expref: -99 pid: 190294 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.349964:0:229334:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.121@tcp lock: ffff9cb454f50e00/0x66475b4e8b79729c lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x66475b4e8b7972a3 expref: -99 pid: 190294 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.349971:0:229334:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.121@tcp lock: ffff9cb454f50e00/0x66475b4e8b79729c lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000400000000 nid: local remote: 0x66475b4e8b7972a3 expref: -99 pid: 190294 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.349978:0:229334:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.202.121@tcp lock: ffff9cb454f50e00/0x66475b4e8b79729c lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x11000400000000 nid: local remote: 0x66475b4e8b7972a3 expref: -99 pid: 190294 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.350010:0:229334:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:0.0:1782488203.350017:0:229334:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.202.121@tcp lock: ffff9cb576a0de00/0x66475b4e8b7973ec lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0x66475b4e8b7973f3 expref: -99 pid: 190294 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.350033:0:229334:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.121@tcp lock: ffff9cb576a0de00/0x66475b4e8b7973ec lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x1000000000000 nid: local remote: 0x66475b4e8b7973f3 expref: -99 pid: 190294 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.350039:0:229334:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.121@tcp lock: ffff9cb576a0de00/0x66475b4e8b7973ec lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x11000400000000 nid: local remote: 0x66475b4e8b7973f3 expref: -99 pid: 190294 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.350044:0:229334:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.202.121@tcp lock: ffff9cb576a0de00/0x66475b4e8b7973ec lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x11000400000000 nid: local remote: 0x66475b4e8b7973f3 expref: -99 pid: 190294 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.350056:0:229334:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:0.0:1782488203.350061:0:229334:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.202.121@tcp lock: ffff9cb57fc71200/0x66475b4e8b796fe7 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x66475b4e8b796fee expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.350067:0:229334:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.121@tcp lock: ffff9cb57fc71200/0x66475b4e8b796fe7 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x66475b4e8b796fee expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.350078:0:229334:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.202.121@tcp lock: ffff9cb57fc71200/0x66475b4e8b796fe7 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 2 type: PLN flags: 0x11000400000000 nid: local remote: 0x66475b4e8b796fee expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.350083:0:229334:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.202.121@tcp lock: ffff9cb57fc71200/0x66475b4e8b796fe7 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 2 type: PLN flags: 0x11000400000000 nid: local remote: 0x66475b4e8b796fee expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.350092:0:229334:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 10000000:01000000:0.0:1782488203.350096:0:229334:0:(mgc_request.c:547:config_log_end()) end config log lustre-MDT0000 (0) 00000020:01000004:0.0:1782488203.350131:0:229334:0:(tgt_mount.c:2112:server_put_super()) stopping lustre-MDT0000 00000020:01000000:0.0:1782488203.350135:0:229334:0:(obd_config.c:2475:class_manual_cleanup()) Manual cleanup of lustre-MDT0000 (flags='F') 00000020:00000080:2.0:1782488203.350138:0:223038:0:(genops.c:865:class_export_destroy()) destroying export ffff9cb560b82800/lustre-MDT0000-lwp-MDT0000_UUID for lustre-MDT0000 00000020:00000080:0.0:1782488203.350140:0:229334:0:(obd_config.c:1470:class_process_config()) processing cmd: cf004 00000020:00000080:0.0:1782488203.350145:0:229334:0:(obd_config.c:955:class_cleanup()) lustre-MDT0000: forcing exports to disconnect: 2/9 00000020:00080000:0.0:1782488203.350150:0:229334:0:(genops.c:1840:print_export_data()) lustre-MDT0000: ACTIVE ffff9cb44a666000 lustre-MDT0000-lwp-OST0001_UUID 0@lo 4 (0 0 0) 0 0 0 0: 0000000000000000 0 stale:0 00000020:00080000:0.0:1782488203.350165:0:229334:0:(genops.c:1840:print_export_data()) lustre-MDT0000: ACTIVE ffff9cb442f38800 lustre-MDT0000-lwp-OST0000_UUID 0@lo 4 (0 0 0) 0 0 0 0: 0000000000000000 0 stale:0 00000020:00080000:0.0:1782488203.350171:0:229334:0:(genops.c:1840:print_export_data()) lustre-MDT0000: UNLINKED ffff9cb551430000 5c4ca15f-f34e-4111-b105-de67ffe3ec92 192.168.202.21@tcp 2 (0 0 0) 1 0 0 0: 0000000000000000 8589967277 stale:0 00000020:00080000:0.0:1782488203.350177:0:229334:0:(genops.c:1574:class_disconnect_exports()) OBD device 5 (ffff9cb57b3eb380) has exports, disconnecting them 00000020:00080000:0.0:1782488203.350181:0:229334:0:(genops.c:1549:class_disconnect_export_list()) lustre-MDT0000: disconnecting export at 0@lo (ffff9cb44a666000), last request at 1782488202 00000020:00000080:0.0:1782488203.350185:0:229334:0:(genops.c:1489:class_disconnect()) disconnect: cookie 0x66475b4e8b79728e 00000100:00080000:0.0:1782488203.350197:0:229334:0:(import.c:65:import_set_state_nolock()) ffff9cb44a661800 : changing import state from FULL to CLOSED 00000020:00080000:0.0:1782488203.350212:0:229334:0:(genops.c:1554:class_disconnect_export_list()) disconnected export at 0@lo (ffff9cb44a666000): rc 0 00000020:00000080:0.0:1782488203.350216:0:229334:0:(genops.c:906:class_export_put()) final put ffff9cb44a666000/lustre-MDT0000-lwp-OST0001_UUID 00000020:00080000:0.0:1782488203.350573:0:229334:0:(genops.c:1549:class_disconnect_export_list()) lustre-MDT0000: disconnecting export at 0@lo (ffff9cb442f38800), last request at 1782488202 00000020:00000080:0.0:1782488203.350578:0:229334:0:(genops.c:1489:class_disconnect()) disconnect: cookie 0x66475b4e8b797145 00000100:00080000:0.0:1782488203.350583:0:229334:0:(import.c:65:import_set_state_nolock()) ffff9cb442f3b000 : changing import state from FULL to CLOSED 00000020:00080000:0.0:1782488203.350591:0:229334:0:(genops.c:1554:class_disconnect_export_list()) disconnected export at 0@lo (ffff9cb442f38800): rc 0 00000020:00000080:0.0:1782488203.350593:0:229334:0:(genops.c:906:class_export_put()) final put ffff9cb442f38800/lustre-MDT0000-lwp-OST0000_UUID 00000020:00000080:2.0:1782488203.350645:0:223038:0:(genops.c:1204:obd_zombie_import_free()) destroying import ffff9cb44108d000 for lustre-MDT0000 00000004:00000080:0.0:1782488203.355188:0:229334:0:(mdd_device.c:2302:mdd_iocontrol()) lustre-MDD0000: cmd=400866e7 len=0 karg=ffffb1d4827a3ae0 00000100:00080000:0.0:1782488203.355260:0:229334:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import lustre-OST0000_UUID INVALID 00000100:00100000:0.0:1782488203.355265:0:229334:0:(client.c:3111:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 8589942199 gen 3 00000004:00080000:0.0:1782488203.355289:0:229334:0:(osp_dev.c:1634:osp_import_event()) got inactive 00000100:00080000:0.0:1782488203.355357:0:229334:0:(pinger.c:400:ptlrpc_pinger_del_import()) removing pingable import lustre-MDT0000-mdtlov_UUID->lustre-OST0000_UUID 00000100:00080000:0.0:1782488203.355383:0:229334:0:(import.c:65:import_set_state_nolock()) ffff9cb579cb3800 lustre-OST0000_UUID: changing import state from FULL to CONNECTING 00000100:00080000:0.0:1782488203.355403:0:229334:0:(import.c:138:ptlrpc_deactivate_import_nolock()) setting import lustre-OST0001_UUID INVALID 00000100:00100000:0.0:1782488203.355407:0:229334:0:(client.c:3111:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 8589941486 gen 3 00000004:00080000:0.0:1782488203.355416:0:229334:0:(osp_dev.c:1634:osp_import_event()) got inactive 00000100:00080000:0.0:1782488203.355477:0:229334:0:(pinger.c:400:ptlrpc_pinger_del_import()) removing pingable import lustre-MDT0000-mdtlov_UUID->lustre-OST0001_UUID 00000100:00080000:0.0:1782488203.355486:0:229334:0:(import.c:65:import_set_state_nolock()) ffff9cb574903000 lustre-OST0001_UUID: changing import state from FULL to CONNECTING 00000020:00080000:0.0:1782488203.355514:0:229334:0:(genops.c:1579:class_disconnect_exports()) OBD device 5 (ffff9cb57b3eb380) has no exports 00010000:00010000:0.0:1782488203.355539:0:191970:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:0.0:1782488203.355547:0:191970:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.202.121@tcp lock: ffff9cb57fc70600/0x66475b4e8b796ff5 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1000400000000 nid: local remote: 0x66475b4e8b796ffc expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.355574:0:191970:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cb57fc70600 already unused, calling callback (ffffffffc1d8bc20) 10000000:00010000:0.0:1782488203.355578:0:191970:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.202.121@tcp lock: ffff9cb57fc70600/0x66475b4e8b796ff5 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1000400000000 nid: local remote: 0x66475b4e8b796ffc expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.355589:0:191970:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.121@tcp lock: ffff9cb57fc70600/0x66475b4e8b796ff5 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1008400000000 nid: local remote: 0x66475b4e8b796ffc expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1782488203.355596:0:191970:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.121@tcp lock: ffff9cb57fc70600/0x66475b4e8b796ff5 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1009400000000 nid: local remote: 0x66475b4e8b796ffc expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1782488203.355602:0:191970:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 10000000:01000000:0.0:1782488203.355620:0:191970:0:(mgc_request.c:944:mgc_blocking_ast()) log lustre-MDT0000: stopping, won't requeue 10000000:01000000:0.0:1782488203.355623:0:191970:0:(mgc_request.c:130:config_log_put()) dropping config log lustre-MDT0000 00010000:00010000:0.0:1782488203.355651:0:191970:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.202.121@tcp lock: ffff9cb57fc70600/0x66475b4e8b796ff5 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x5809400000020 nid: local remote: 0x66475b4e8b796ffc expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.355657:0:191970:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1782488203.355675:0:191970:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.202.121@tcp lock: ffff9cb57fc70600/0x66475b4e8b796ff5 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x5809400000020 nid: local remote: 0x66475b4e8b796ffc expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.355682:0:191970:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.121@tcp lock: ffff9cb57fc70600/0x66475b4e8b796ff5 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x5809400000020 nid: local remote: 0x66475b4e8b796ffc expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.355696:0:191970:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:0.0:1782488203.355699:0:191970:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.202.121@tcp lock: ffff9cb454f50e00/0x66475b4e8b79729c lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000400000000 nid: local remote: 0x66475b4e8b7972a3 expref: -99 pid: 190294 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.355705:0:191970:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cb454f50e00 already unused, calling callback (ffffffffc1d8bc20) 10000000:00010000:0.0:1782488203.355708:0:191970:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.202.121@tcp lock: ffff9cb454f50e00/0x66475b4e8b79729c lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1000400000000 nid: local remote: 0x66475b4e8b7972a3 expref: -99 pid: 190294 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.355722:0:191970:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.121@tcp lock: ffff9cb454f50e00/0x66475b4e8b79729c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1008400000000 nid: local remote: 0x66475b4e8b7972a3 expref: -99 pid: 190294 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1782488203.355728:0:191970:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.121@tcp lock: ffff9cb454f50e00/0x66475b4e8b79729c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x1009400000000 nid: local remote: 0x66475b4e8b7972a3 expref: -99 pid: 190294 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1782488203.355733:0:191970:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 10000000:01000000:0.0:1782488203.355735:0:191970:0:(mgc_request.c:944:mgc_blocking_ast()) log lustre-mdtir: stopping, won't requeue 10000000:01000000:0.0:1782488203.355737:0:191970:0:(mgc_request.c:130:config_log_put()) dropping config log lustre-mdtir 00010000:00010000:0.0:1782488203.355752:0:191970:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.202.121@tcp lock: ffff9cb454f50e00/0x66475b4e8b79729c lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5809400000020 nid: local remote: 0x66475b4e8b7972a3 expref: -99 pid: 190294 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.355757:0:191970:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1782488203.355770:0:191970:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.202.121@tcp lock: ffff9cb454f50e00/0x66475b4e8b79729c lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5809400000020 nid: local remote: 0x66475b4e8b7972a3 expref: -99 pid: 190294 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.355777:0:191970:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.121@tcp lock: ffff9cb454f50e00/0x66475b4e8b79729c lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 2 type: PLN flags: 0x5809400000020 nid: local remote: 0x66475b4e8b7972a3 expref: -99 pid: 190294 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.355791:0:191970:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1782488203.355795:0:191970:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.202.121@tcp lock: ffff9cb576a0de00/0x66475b4e8b7973ec lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x1000400000000 nid: local remote: 0x66475b4e8b7973f3 expref: -99 pid: 190294 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.355801:0:191970:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cb576a0de00 already unused, calling callback (ffffffffc1d8bc20) 10000000:00010000:0.0:1782488203.355803:0:191970:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.202.121@tcp lock: ffff9cb576a0de00/0x66475b4e8b7973ec lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x1000400000000 nid: local remote: 0x66475b4e8b7973f3 expref: -99 pid: 190294 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.355810:0:191970:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.121@tcp lock: ffff9cb576a0de00/0x66475b4e8b7973ec lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x1008400000000 nid: local remote: 0x66475b4e8b7973f3 expref: -99 pid: 190294 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1782488203.355828:0:191970:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.121@tcp lock: ffff9cb576a0de00/0x66475b4e8b7973ec lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x1009400000000 nid: local remote: 0x66475b4e8b7973f3 expref: -99 pid: 190294 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1782488203.355833:0:191970:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 10000000:01000000:0.0:1782488203.355850:0:191970:0:(mgc_request.c:944:mgc_blocking_ast()) log params: stopping, won't requeue 10000000:01000000:0.0:1782488203.355853:0:191970:0:(mgc_request.c:130:config_log_put()) dropping config log params 00010000:00010000:0.0:1782488203.355868:0:191970:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.202.121@tcp lock: ffff9cb576a0de00/0x66475b4e8b7973ec lrc: 2/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x5809400000020 nid: local remote: 0x66475b4e8b7973f3 expref: -99 pid: 190294 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.355874:0:191970:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1782488203.355880:0:191970:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.202.121@tcp lock: ffff9cb576a0de00/0x66475b4e8b7973ec lrc: 1/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x5809400000020 nid: local remote: 0x66475b4e8b7973f3 expref: -99 pid: 190294 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.355886:0:191970:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.121@tcp lock: ffff9cb576a0de00/0x66475b4e8b7973ec lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 4 type: PLN flags: 0x5809400000020 nid: local remote: 0x66475b4e8b7973f3 expref: -99 pid: 190294 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.355895:0:191970:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1782488203.355898:0:191970:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.202.121@tcp lock: ffff9cb57fc71200/0x66475b4e8b796fe7 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 2 type: PLN flags: 0x1000400000000 nid: local remote: 0x66475b4e8b796fee expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.355904:0:191970:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff9cb57fc71200 already unused, calling callback (ffffffffc1d8bc20) 10000000:00010000:0.0:1782488203.355907:0:191970:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.202.121@tcp lock: ffff9cb57fc71200/0x66475b4e8b796fe7 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 2 type: PLN flags: 0x1000400000000 nid: local remote: 0x66475b4e8b796fee expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.355913:0:191970:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.202.121@tcp lock: ffff9cb57fc71200/0x66475b4e8b796fe7 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 2 type: PLN flags: 0x1008400000000 nid: local remote: 0x66475b4e8b796fee expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1782488203.355918:0:191970:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.202.121@tcp lock: ffff9cb57fc71200/0x66475b4e8b796fe7 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 2 type: PLN flags: 0x1009400000000 nid: local remote: 0x66475b4e8b796fee expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1782488203.355934:0:191970:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x5:0x0].0x0 (lustre) 10000000:01000000:0.0:1782488203.355936:0:191970:0:(mgc_request.c:944:mgc_blocking_ast()) log lustre-barrier: stopping, won't requeue 10000000:01000000:0.0:1782488203.355938:0:191970:0:(mgc_request.c:130:config_log_put()) dropping config log lustre-barrier 00010000:00010000:0.0:1782488203.355951:0:191970:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.202.121@tcp lock: ffff9cb57fc71200/0x66475b4e8b796fe7 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 2 type: PLN flags: 0x5809400000020 nid: local remote: 0x66475b4e8b796fee expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.355956:0:191970:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1782488203.355962:0:191970:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.202.121@tcp lock: ffff9cb57fc71200/0x66475b4e8b796fe7 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 2 type: PLN flags: 0x5809400000020 nid: local remote: 0x66475b4e8b796fee expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.355967:0:191970:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.202.121@tcp lock: ffff9cb57fc71200/0x66475b4e8b796fe7 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 2 type: PLN flags: 0x5809400000020 nid: local remote: 0x66475b4e8b796fee expref: -99 pid: 190278 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1782488203.355977:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488203.355979:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488203.355981:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488203.355988:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1782488203.356153:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579575500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300134400:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1782488203.356214:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488203.356258:0:3315:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1782488203.356263:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579576300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300134784:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1782488203.356283:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1782488203.356301:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1782488203.356310:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579577480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3313:1869066300134144:0@lo:9:umount.0 00000100:00100000:0.0:1782488203.356336:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488203.356353:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579574000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3313:1869066300134272:0@lo:9:umount.0 00000100:00100000:0.0:1782488203.356372:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488203.356407:0:213226:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300134144 00000100:00100000:0.0:1782488203.356414:0:213226:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300134144 00000100:00100000:0.0:1782488203.356449:0:213226:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300134272 00000100:00100000:0.0:1782488203.356451:0:213226:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300134272 00000100:00100000:0.0:1782488203.356471:0:213226:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 20825 00000100:00100000:0.0:1782488203.356479:0:213226:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579574e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3313:x1869066300134144:12345-0@lo:9:umount.0 00000020:00000080:0.0:1782488203.356502:0:213226:0:(genops.c:1489:class_disconnect()) disconnect: cookie 0x66475b4e8b797153 00000100:00080000:0.0:1782488203.356511:0:213226:0:(import.c:65:import_set_state_nolock()) ffff9cb55840f800 : changing import state from FULL to CLOSED 00000100:00100000:1.0:1782488203.356689:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579574700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300134528:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1782488203.356717:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488203.356738:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1782488203.356744:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579576680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300134656:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1782488203.356763:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488203.356792:0:192528:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300134400 00000100:00100000:1.0:1782488203.356796:0:192528:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300134400 00000100:00100000:1.0:1782488203.356823:0:192528:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300134784 00000100:00100000:1.0:1782488203.356826:0:192528:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300134784 00000100:00100000:1.0:1782488203.356856:0:192528:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300134528 00000100:00100000:1.0:1782488203.356858:0:192528:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300134528 00000100:00100000:1.0:1782488203.356875:0:192528:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300134656 00000100:00100000:1.0:1782488203.356877:0:192528:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300134656 00000100:00100000:1.0:1782488203.356889:0:192528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1223 00000100:00100000:1.0:1782488203.356894:0:192528:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579577b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:6385baa4-edbe-4eae-af92-80250729af95+21:3315:x1869066300134400:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1782488203.356905:0:192528:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1782488203.356923:0:192528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff9cb55919ca00/0x66475b4e8b796ffc lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x44801000000000 nid: 0@lo remote: 0x66475b4e8b796ff5 expref: 21 pid: 190292 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782488203.356936:0:192528:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1782488203.356969:0:192528:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579577b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:6385baa4-edbe-4eae-af92-80250729af95+20:3315:x1869066300134400:12345-0@lo:103:ldlm_bl.0 Request processed in 74us (755us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488203.356976:0:192528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1223 00000100:00100000:1.0:1782488203.356995:0:192528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1224 00000100:00100000:1.0:1782488203.357001:0:192528:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57fc94c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:6385baa4-edbe-4eae-af92-80250729af95+19:3315:x1869066300134784:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1782488203.357008:0:192528:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1782488203.357016:0:192528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff9cb454f51a00/0x66475b4e8b796fee lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 3 type: PLN flags: 0x44801000000000 nid: 0@lo remote: 0x66475b4e8b796fe7 expref: 19 pid: 190291 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782488203.357027:0:192528:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1782488203.357047:0:192528:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57fc94c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:6385baa4-edbe-4eae-af92-80250729af95+18:3315:x1869066300134784:12345-0@lo:103:ldlm_bl.0 Request processed in 46us (763us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488203.357053:0:192528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1224 00000100:00100000:1.0:1782488203.357087:0:192528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1225 00000100:00100000:1.0:1782488203.357092:0:192528:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb450349880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:6385baa4-edbe-4eae-af92-80250729af95+17:3316:x1869066300134528:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1782488203.357098:0:192528:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1782488203.357105:0:192528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff9cb57fc71c00/0x66475b4e8b7972a3 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x44801000000000 nid: 0@lo remote: 0x66475b4e8b79729c expref: 17 pid: 190290 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782488203.357113:0:192528:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1782488203.357136:0:192528:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb450349880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:6385baa4-edbe-4eae-af92-80250729af95+16:3316:x1869066300134528:12345-0@lo:103:ldlm_bl.0 Request processed in 44us (418us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488203.357141:0:192528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1225 00000100:00100000:1.0:1782488203.357155:0:192528:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1226 00000100:00100000:1.0:1782488203.357166:0:192528:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45034b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:6385baa4-edbe-4eae-af92-80250729af95+15:3316:x1869066300134656:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1782488203.357173:0:192528:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1782488203.357180:0:192528:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff9cb57fc70800/0x66475b4e8b7973f3 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 5 type: PLN flags: 0x44801000000000 nid: 0@lo remote: 0x66475b4e8b7973ec expref: 15 pid: 190292 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1782488203.357188:0:192528:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1782488203.357209:0:192528:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45034b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300134656:12345-0@lo:103:ldlm_bl.0 Request processed in 43us (445us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488203.357214:0:192528:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1226 00000100:00100000:1.0:1782488203.357256:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579576680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300134656:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1782488203.357264:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579574700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300134528:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1782488203.357288:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579576300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300134784:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1782488203.357296:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579575500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300134400:0@lo:103:ldlm_bl.0 00000020:00000080:2.0:1782488203.357560:0:223038:0:(genops.c:865:class_export_destroy()) destroying export ffff9cb44a666000/lustre-MDT0000-lwp-OST0001_UUID for lustre-MDT0000 00000020:00000080:2.0:1782488203.357732:0:223038:0:(genops.c:865:class_export_destroy()) destroying export ffff9cb442f38800/lustre-MDT0000-lwp-OST0000_UUID for lustre-MDT0000 00000020:00000080:2.0:1782488203.357756:0:223038:0:(genops.c:1204:obd_zombie_import_free()) destroying import ffff9cb44a661800 for lustre-MDT0000 00000020:00000080:2.0:1782488203.357761:0:223038:0:(genops.c:1204:obd_zombie_import_free()) destroying import ffff9cb442f3b000 for lustre-MDT0000 00000100:00100000:2.0:1782488203.359747:0:213693:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 20826 00000100:00100000:2.0:1782488203.359768:0:213693:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+5:3313:x1869066300134272:12345-0@lo:9:umount.0 00000020:00000080:2.0:1782488203.359797:0:213693:0:(genops.c:1489:class_disconnect()) disconnect: cookie 0x66475b4e8b7972aa 00000100:00080000:2.0:1782488203.359806:0:213693:0:(import.c:65:import_set_state_nolock()) ffff9cb44a665800 : changing import state from FULL to CLOSED 00000001:00000004:2.0:1782488203.359902:0:213693:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0001_UUID: mount_count is 2, last_transno is 8589941508 00000100:00100000:2.0:1782488203.360282:0:213693:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579576d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_019:lustre-MDT0000-mdtlov_UUID+2:3313:x1869066300134272:12345-0@lo:9:umount.0 Request processed in 523us (3906us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488203.360290:0:213693:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 20826 00010000:00010000:2.0:1782488203.360486:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00080000:2.0:1782488203.360792:0:229334:0:(genops.c:1579:class_disconnect_exports()) OBD device 5 (ffff9cb57b3eb380) has no exports 00000001:00000004:0.0:1782488203.361326:0:213226:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0000_UUID: mount_count is 2, last_transno is 8589942216 00000100:00100000:0.0:1782488203.361576:0:213226:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579574e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+2:3313:x1869066300134144:12345-0@lo:9:umount.0 Request processed in 5096us (5237us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488203.361585:0:213226:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 20825 00010000:00010000:0.0:1782488203.361627:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488203.361660:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579cb3800 lustre-OST0000_UUID: changing import state from CONNECTING to CLOSED 00000004:00080000:0.0:1782488203.361675:0:3313:0:(osp_dev.c:1618:osp_import_event()) got disconnected 00000004:00080000:0.0:1782488203.361676:0:3313:0:(osp_dev.c:1634:osp_import_event()) got inactive 00000100:00100000:0.0:1782488203.361680:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579577480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3313:1869066300134144:0@lo:9:umount.0 00000100:00080000:0.0:1782488203.361693:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb574903000 lustre-OST0001_UUID: changing import state from CONNECTING to CLOSED 00000004:00080000:0.0:1782488203.361701:0:3313:0:(osp_dev.c:1618:osp_import_event()) got disconnected 00000004:00080000:0.0:1782488203.361702:0:3313:0:(osp_dev.c:1634:osp_import_event()) got inactive 00000100:00100000:0.0:1782488203.361704:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579574000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-mdtlov_UUID:3313:1869066300134272:0@lo:9:umount.0 00010000:00010000:0.0:1782488203.361735:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:2.0:1782488203.426952:0:229334:0:(genops.c:1489:class_disconnect()) disconnect: cookie 0x66475b4e8b797042 00000020:00000080:2.0:1782488203.426967:0:229334:0:(genops.c:906:class_export_put()) final put ffff9cb579cb6000/lustre-QMT0000_UUID 00000020:01000000:2.0:1782488203.426993:0:229334:0:(obd_config.c:2475:class_manual_cleanup()) Manual cleanup of lustre-QMT0000 (flags='') 00000020:00000080:1.0:1782488203.427664:0:229334:0:(obd_config.c:1470:class_process_config()) processing cmd: cf004 00000020:00000080:1.0:1782488203.427678:0:229334:0:(obd_config.c:955:class_cleanup()) lustre-QMT0000: forcing exports to disconnect: 0/2 00000020:00080000:1.0:1782488203.427683:0:229334:0:(genops.c:1579:class_disconnect_exports()) OBD device 7 (ffff9cb57ed64d40) has no exports 00000020:00000080:3.0:1782488203.427877:0:186303:0:(genops.c:865:class_export_destroy()) destroying export ffff9cb579cb6000/lustre-QMT0000_UUID for lustre-QMT0000 00000020:00000080:1.0:1782488203.435142:0:229334:0:(genops.c:1489:class_disconnect()) disconnect: cookie 0x66475b4e8b79703b 00000020:00000080:1.0:1782488203.435157:0:229334:0:(genops.c:906:class_export_put()) final put ffff9cb57493c000/lustre-MDT0000-osd_UUID 00000020:00000080:1.0:1782488203.435665:0:229334:0:(obd_config.c:1470:class_process_config()) processing cmd: cf002 00000020:00000080:1.0:1782488203.435678:0:229334:0:(obd_config.c:883:class_detach()) detach on obd lustre-QMT0000 (uuid lustre-QMT0000_UUID) 00000020:00000080:1.0:1782488203.435681:0:229334:0:(genops.c:906:class_export_put()) final put ffff9cb559267800/lustre-QMT0000_UUID 00000020:00000080:1.0:1782488203.435685:0:229334:0:(genops.c:865:class_export_destroy()) destroying export ffff9cb559267800/lustre-QMT0000_UUID for lustre-QMT0000 00000020:01000000:1.0:1782488203.435695:0:229334:0:(genops.c:441:class_free_dev()) finishing cleanup of obd lustre-QMT0000 (lustre-QMT0000_UUID) 00000020:00000080:3.0:1782488203.435980:0:186303:0:(genops.c:865:class_export_destroy()) destroying export ffff9cb57493c000/lustre-MDT0000-osd_UUID for lustre-MDT0000-osd 00000001:00100000:2.0:1782488203.623141:0:6622:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of lustre-MDT0000-mdtlov_UUID 00000020:00000080:2.0:1782488203.623152:0:6622:0:(genops.c:906:class_export_put()) final put ffff9cb55840d000/lustre-MDT0000-mdtlov_UUID 00000020:00000080:2.0:1782488203.623196:0:186303:0:(genops.c:865:class_export_destroy()) destroying export ffff9cb55840d000/lustre-MDT0000-mdtlov_UUID for lustre-OST0000 00000020:00000080:2.0:1782488203.623402:0:186303:0:(genops.c:1204:obd_zombie_import_free()) destroying import ffff9cb55840f800 for lustre-OST0000 00010000:00010000:1.0:1782488203.906960:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488204.929772:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488205.954351:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488206.989072:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1782488207.170481:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442933480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300134912:0@lo:400:kworker.0 00000100:00100000:3.0:1782488207.170571:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488207.170648:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442933b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3316:1869066300135040:0@lo:400:kworker.0 00000100:00100000:3.0:1782488207.170675:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488207.170694:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1782488207.170699:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442933100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3316:1869066300135168:0@lo:400:kworker.0 00000100:00100000:3.0:1782488207.170718:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488207.170750:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300134912 00000100:00100000:3.0:1782488207.170758:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300134912 00000100:00100000:3.0:1782488207.170826:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 584 00000100:00100000:3.0:1782488207.170834:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f341450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300134912:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782488207.170891:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f341450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300134912:12345-0@lo:400:kworker.0 Request processed in 57us (320us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488207.170898:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 584 00000100:00100000:3.0:1782488207.170952:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300135040 00000100:00100000:3.0:1782488207.170956:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300135040 00000100:00100000:3.0:1782488207.170986:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300135168 00000100:00100000:3.0:1782488207.170988:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300135168 00000100:00100000:3.0:1782488207.171003:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14004 00000100:00100000:3.0:1782488207.171008:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb442932680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3316:x1869066300135040:12345-0@lo:400:kworker.0 00000020:00080000:3.0:1782488207.171014:0:213684:0:(tgt_handler.c:807:tgt_request_handle()) operation 400 on unconnected OST from 12345-0@lo 00000100:00100000:3.0:1782488207.171040:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb442932680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3316:x1869066300135040:12345-0@lo:400:kworker.0 Request processed in 31us (364us total) trans 0 rc -107/-107 00000100:00100000:3.0:1782488207.171046:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14004 00000100:00100000:3.0:1782488207.171055:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14005 00000100:00100000:3.0:1782488207.171058:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb442930e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3316:x1869066300135168:12345-0@lo:400:kworker.0 00000020:00080000:3.0:1782488207.171076:0:213684:0:(tgt_handler.c:807:tgt_request_handle()) operation 400 on unconnected OST from 12345-0@lo 00000100:00100000:3.0:1782488207.171093:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb442930e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3316:x1869066300135168:12345-0@lo:400:kworker.0 Request processed in 34us (375us total) trans 0 rc -107/-107 00000100:00100000:3.0:1782488207.171099:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14005 00000100:00080000:3.0:1782488207.171156:0:3316:0:(recover.c:205:ptlrpc_request_handle_notconn()) import lustre-MDT0000-lwp-OST0001 of lustre-MDT0000_UUID@0@lo abruptly disconnected: reconnecting 00000100:00080000:3.0:1782488207.171161:0:3316:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:3.0:1782488207.171165:0:3316:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete 00000100:00100000:1.0:1782488207.173393:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442933480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300134912:0@lo:400:kworker.0 00000100:00080000:3.0:1782488207.188983:0:3316:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1782488207.188998:0:3316:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 6665 00000100:00080000:3.0:1782488207.189008:0:3316:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:3.0:1782488207.189049:0:3316:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb442931c00 x1869066300135296/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1782488207.189080:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442933100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3316:1869066300135168:0@lo:400:kworker.0 00000100:00080000:3.0:1782488207.189116:0:3316:0:(recover.c:205:ptlrpc_request_handle_notconn()) import lustre-MDT0000-lwp-OST0000 of lustre-MDT0000_UUID@0@lo abruptly disconnected: reconnecting 00000100:00080000:3.0:1782488207.189120:0:3316:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:3.0:1782488207.189123:0:3316:0:(import.c:182:ptlrpc_set_import_discon()) lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:3.0:1782488207.189127:0:3316:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1782488207.189130:0:3316:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 6655 00000100:00080000:3.0:1782488207.189134:0:3316:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:3.0:1782488207.189145:0:3316:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb442933480 x1869066300135424/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1782488207.189155:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442933b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3316:1869066300135040:0@lo:400:kworker.0 00000100:00100000:1.0:1782488207.189527:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442931c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300135296:0@lo:38:ptlrpcd_00_02.0 00000100:00100000:1.0:1782488207.189629:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488207.189659:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442933480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300135424:0@lo:38:ptlrpcd_00_02.0 00000100:00100000:1.0:1782488207.189679:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488207.190080:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300135296 00000100:00100000:1.0:1782488207.190088:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300135296 00000100:00100000:1.0:1782488207.190134:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300135424 00000100:00100000:1.0:1782488207.190136:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300135424 00000100:00100000:1.0:1782488207.190153:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14006 00000100:00100000:1.0:1782488207.190160:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb450348a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300135296:12345-0@lo:38:ptlrpcd_00_02.0 00010000:02000000:1.0:1782488207.190178:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488207.190374:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14007 00000100:00100000:3.0:1782488207.190394:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb450348380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300135424:12345-0@lo:38:ptlrpcd_00_02.0 00010000:02000000:3.0:1782488207.190402:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488207.190466:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb450348380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300135424:12345-0@lo:38:ptlrpcd_00_02.0 Request processed in 71us (786us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488207.190976:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14007 00000100:00080000:3.0:1782488207.192720:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1782488207.192728:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:3.0:1782488207.192733:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442933480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300135424:0@lo:38:ptlrpcd_00_02.0 00000100:00080000:2.0:1782488207.210492:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488207.210503:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488207.210508:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442931c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300135296:0@lo:38:ptlrpcd_00_02.0 00000100:00100000:1.0:1782488207.210875:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb450348a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300135296:12345-0@lo:38:ptlrpcd_00_02.0 Request processed in 20255us (20787us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488207.210951:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14006 00010000:00010000:1.0:1782488208.014294:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00100000:3.0:1782488208.306279:0:7511:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of lustre-MDT0000-mdtlov_UUID 00000020:00000080:3.0:1782488208.306348:0:7511:0:(genops.c:906:class_export_put()) final put ffff9cb44a663800/lustre-MDT0000-mdtlov_UUID 00000020:00000080:0.0:1782488208.314543:0:186303:0:(genops.c:865:class_export_destroy()) destroying export ffff9cb44a663800/lustre-MDT0000-mdtlov_UUID for lustre-OST0001 00000020:00000080:0.0:1782488208.314628:0:186303:0:(genops.c:1204:obd_zombie_import_free()) destroying import ffff9cb44a665800 for lustre-OST0001 00010000:00010000:0.0:1782488209.027429:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488210.069693:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488211.075832:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488212.104151:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1782488212.289815:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1782488212.289823:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1782488212.289827:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1782488212.289836:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7834 00000100:00080000:3.0:1782488212.289842:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 5s 00000100:00080000:3.0:1782488212.289849:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:3.0:1782488212.289873:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb442933b80 x1869066300135680/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1782488212.289890:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1782488212.289894:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1782488212.289895:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1782488212.289899:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7834 00000100:00080000:3.0:1782488212.289901:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 5s 00000100:00080000:3.0:1782488212.289905:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:3.0:1782488212.289916:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb442933100 x1869066300135808/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488212.290054:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442933480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300135552:0@lo:400:kworker.0 00000100:00100000:0.0:1782488212.290150:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488212.290345:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442933b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300135680:0@lo:38:kworker.0 00000100:00100000:3.0:1782488212.290440:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488212.290463:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442933100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300135808:0@lo:38:kworker.0 00000100:00100000:3.0:1782488212.290499:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488212.290550:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300135680 00000100:00100000:3.0:1782488212.290558:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300135680 00000100:00100000:3.0:1782488212.290584:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300135808 00000100:00100000:3.0:1782488212.290586:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300135808 00000100:00100000:3.0:1782488212.290600:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14008 00000100:00100000:3.0:1782488212.291037:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb442931c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300135680:12345-0@lo:38:kworker.0 00010000:02000000:3.0:1782488212.291056:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488212.291439:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300135552 00000100:00100000:0.0:1782488212.291457:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300135552 00000100:00100000:0.0:1782488212.291500:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 585 00000100:00100000:0.0:1782488212.291507:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f340850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300135552:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488212.291574:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f340850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300135552:12345-0@lo:400:kworker.0 Request processed in 68us (1424us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488212.291580:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 585 00000100:00100000:0.0:1782488212.295184:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442933480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300135552:0@lo:400:kworker.0 00000100:00100000:1.0:1782488212.301414:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14009 00000100:00100000:1.0:1782488212.301429:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455a2f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300135808:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488212.301452:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488212.301551:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455a2f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300135808:12345-0@lo:38:kworker.0 Request processed in 122us (11049us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488212.301559:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14009 00000100:00080000:1.0:1782488212.301636:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488212.301642:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488212.301646:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442933100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300135808:0@lo:38:kworker.0 00000100:00100000:3.0:1782488212.310160:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb442931c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300135680:12345-0@lo:38:kworker.0 Request processed in 19544us (19712us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488212.310176:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14008 00000100:00080000:3.0:1782488212.311576:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1782488212.311585:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:3.0:1782488212.311589:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442933b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300135680:0@lo:38:kworker.0 00010000:00010000:0.0:1782488213.122951:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488214.145858:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488215.179125:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488216.196660:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488217.217729:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1782488217.409839:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488217.409847:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488217.409851:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488217.409859:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7839 00000100:00080000:1.0:1782488217.409863:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 10s 00000100:00080000:1.0:1782488217.409935:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488217.409958:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb450348000 x1869066300136064/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1782488217.409975:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488217.409978:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488217.409980:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488217.409983:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7839 00000100:00080000:1.0:1782488217.409987:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 10s 00000100:00080000:1.0:1782488217.409990:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488217.410002:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb45034b800 x1869066300136192/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1782488217.410027:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1782488217.410043:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45034b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300135936:0@lo:400:kworker.0 00000100:00100000:3.0:1782488217.410145:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb450348000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300136064:0@lo:38:kworker.0 00000100:00100000:1.0:1782488217.412930:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488217.413281:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300135936 00000100:00100000:2.0:1782488217.413291:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300135936 00000100:00100000:2.0:1782488217.413337:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 586 00000100:00100000:2.0:1782488217.413345:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f342450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300135936:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488217.413404:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f342450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300135936:12345-0@lo:400:kworker.0 Request processed in 60us (479us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488217.413412:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 586 00000100:00100000:2.0:1782488217.413500:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45034b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300135936:0@lo:400:kworker.0 00000100:00100000:3.0:1782488217.422705:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488217.422801:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300136064 00000100:00100000:2.0:1782488217.422810:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300136064 00000100:00100000:3.0:1782488217.422810:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45034b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300136192:0@lo:38:kworker.0 00000100:00100000:3.0:1782488217.422866:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488217.422872:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14010 00000100:00100000:2.0:1782488217.422879:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455a2ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300136064:12345-0@lo:38:kworker.0 00000100:00100000:3.0:1782488217.422886:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300136192 00000100:00100000:3.0:1782488217.422889:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300136192 00010000:02000000:2.0:1782488217.422898:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488217.422908:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14011 00000100:00100000:3.0:1782488217.422912:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455a2c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300136192:12345-0@lo:38:kworker.0 00010000:02000000:3.0:1782488217.422934:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488217.422972:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455a2c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300136192:12345-0@lo:38:kworker.0 Request processed in 58us (104us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488217.422980:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14011 00000100:00080000:3.0:1782488217.423017:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1782488217.423022:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:3.0:1782488217.423025:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45034b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300136192:0@lo:38:kworker.0 00000100:00100000:2.0:1782488217.432259:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455a2ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300136064:12345-0@lo:38:kworker.0 Request processed in 9371us (9553us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488217.432275:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14010 00000100:00080000:3.0:1782488217.436066:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1782488217.436077:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:3.0:1782488217.436082:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb450348000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300136064:0@lo:38:kworker.0 00000020:02000400:3.0:1782488217.922117:0:229334:0:(genops.c:1873:obd_exports_barrier()) lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 5. Is it stuck? 00000020:02000400:3.0:1782488217.936522:0:229334:0:(genops.c:1840:print_export_data()) lustre-MDT0000: UNLINKED ffff9cb551430000 5c4ca15f-f34e-4111-b105-de67ffe3ec92 192.168.202.21@tcp 2 (0 0 0) 1 0 0 0: 0000000000000000 8589967277 stale:0 00010000:00010000:0.0:1782488218.246696:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488219.275299:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488220.292800:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488221.315363:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488222.347797:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1782488222.529767:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488222.529775:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488222.529780:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488222.529790:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7844 00000100:00080000:1.0:1782488222.529829:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 15s 00000100:00080000:1.0:1782488222.529837:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488222.529864:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb450348700 x1869066300136448/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1782488222.529891:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488222.529903:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488222.529905:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488222.529910:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7844 00000100:00080000:1.0:1782488222.529914:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 15s 00000100:00080000:1.0:1782488222.529919:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488222.529934:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb45034a680 x1869066300136576/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1782488222.529964:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1782488222.529982:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb450349500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300136320:0@lo:400:kworker.0 00000100:00100000:1.0:1782488222.530059:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488222.530383:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb450348700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300136448:0@lo:38:kworker.0 00000100:00100000:3.0:1782488222.530424:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488222.530450:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45034a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300136576:0@lo:38:kworker.0 00000100:00100000:3.0:1782488222.530476:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488222.530516:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300136320 00000100:00100000:3.0:1782488222.530523:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300136320 00000100:00100000:3.0:1782488222.530586:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 587 00000100:00100000:3.0:1782488222.530596:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f341050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300136320:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782488222.530665:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f341050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300136320:12345-0@lo:400:kworker.0 Request processed in 70us (606us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488222.530696:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 587 00000100:00100000:3.0:1782488222.530729:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300136448 00000100:00100000:3.0:1782488222.530732:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300136448 00000100:00100000:3.0:1782488222.530747:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300136576 00000100:00100000:3.0:1782488222.530749:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300136576 00000100:00100000:3.0:1782488222.530761:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14012 00000100:00100000:3.0:1782488222.530766:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455a2f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300136448:12345-0@lo:38:kworker.0 00010000:02000000:3.0:1782488222.530783:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488222.532849:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455a2f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300136448:12345-0@lo:38:kworker.0 Request processed in 2073us (2415us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488222.532864:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14012 00000100:00100000:3.0:1782488222.532896:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14013 00000100:00100000:3.0:1782488222.532903:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455a2c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300136576:12345-0@lo:38:kworker.0 00010000:02000000:3.0:1782488222.532919:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488222.532944:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455a2c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300136576:12345-0@lo:38:kworker.0 Request processed in 40us (2469us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488222.532949:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14013 00000100:00080000:3.0:1782488222.533011:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1782488222.533017:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:3.0:1782488222.533021:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb450348700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300136448:0@lo:38:kworker.0 00000100:00080000:3.0:1782488222.533031:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1782488222.533034:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:3.0:1782488222.533037:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45034a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300136576:0@lo:38:kworker.0 00000100:00100000:1.0:1782488222.533800:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb450349500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300136320:0@lo:400:kworker.0 00010000:00010000:0.0:1782488223.373769:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488224.385808:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488225.411080:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488226.434374:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488227.460280:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1782488227.650913:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1782488227.650921:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1782488227.650969:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1782488227.650978:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7849 00000100:00080000:3.0:1782488227.650992:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1782488227.651000:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:3.0:1782488227.651024:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb455a2d500 x1869066300136832/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1782488227.651042:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1782488227.651046:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1782488227.651047:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1782488227.651051:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7849 00000100:00080000:3.0:1782488227.651054:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1782488227.651057:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:3.0:1782488227.651068:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb455a2c700 x1869066300136960/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1782488227.651102:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1782488227.651120:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb455a2f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300136704:0@lo:400:kworker.0 00000100:00100000:3.0:1782488227.651193:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488227.651249:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb455a2d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300136832:0@lo:38:kworker.0 00000100:00100000:3.0:1782488227.651277:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488227.651297:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb455a2c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300136960:0@lo:38:kworker.0 00000100:00100000:3.0:1782488227.651318:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488227.651350:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300136704 00000100:00100000:3.0:1782488227.651356:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300136704 00000100:00100000:3.0:1782488227.651399:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 588 00000100:00100000:3.0:1782488227.651418:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f79c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300136704:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782488227.651480:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f79c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300136704:12345-0@lo:400:kworker.0 Request processed in 63us (286us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488227.651489:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 588 00000100:00100000:3.0:1782488227.651521:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300136832 00000100:00100000:3.0:1782488227.651525:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300136832 00000100:00100000:3.0:1782488227.651543:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300136960 00000100:00100000:3.0:1782488227.651546:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300136960 00000100:00100000:3.0:1782488227.651560:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14014 00000100:00100000:3.0:1782488227.651565:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455a2e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300136832:12345-0@lo:38:kworker.0 00010000:02000000:3.0:1782488227.651578:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488227.653151:0:213935:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14015 00000100:00100000:1.0:1782488227.653164:0:213935:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455a2e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300136960:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488227.653183:0:213935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488227.653254:0:213935:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455a2e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300136960:12345-0@lo:38:kworker.0 Request processed in 90us (1934us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488227.653261:0:213935:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14015 00000100:00100000:1.0:1782488227.653793:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb455a2f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300136704:0@lo:400:kworker.0 00000100:00080000:1.0:1782488227.653840:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488227.653859:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488227.653862:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb455a2c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300136960:0@lo:38:kworker.0 00000100:00100000:3.0:1782488227.671264:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455a2e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300136832:12345-0@lo:38:kworker.0 Request processed in 19686us (19975us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488227.671280:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14014 00000100:00080000:3.0:1782488227.671342:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1782488227.671348:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:3.0:1782488227.671352:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb455a2d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300136832:0@lo:38:kworker.0 00010000:00010000:0.0:1782488228.481762:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488229.510859:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488230.532654:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488231.558798:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488232.581206:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488232.771347:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488232.771355:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488232.771373:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488232.771381:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7855 00000100:00080000:2.0:1782488232.771386:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488232.771404:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488232.771431:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff3b80 x1869066300137216/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488232.771447:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488232.771451:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488232.771453:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488232.771456:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7855 00000100:00080000:2.0:1782488232.771458:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488232.771462:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488232.771475:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff0e00 x1869066300137344/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1782488232.772219:0:3315:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1782488232.772239:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300137088:0@lo:400:kworker.0 00000100:00100000:3.0:1782488232.772310:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488232.772373:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300137216:0@lo:38:kworker.0 00000100:00100000:3.0:1782488232.772403:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488232.772423:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300137344:0@lo:38:kworker.0 00000100:00100000:3.0:1782488232.772446:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488232.772476:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300137088 00000100:00100000:3.0:1782488232.772534:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300137088 00000100:00100000:3.0:1782488232.772591:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 589 00000100:00100000:3.0:1782488232.772601:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f79c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300137088:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782488232.772862:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f79c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300137088:12345-0@lo:400:kworker.0 Request processed in 261us (550us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488232.772869:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 589 00000100:00100000:3.0:1782488232.772902:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300137216 00000100:00100000:3.0:1782488232.772905:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300137216 00000100:00100000:3.0:1782488232.772922:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300137344 00000100:00100000:3.0:1782488232.772924:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300137344 00000100:00100000:3.0:1782488232.772937:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14016 00000100:00100000:3.0:1782488232.772942:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455a2d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300137216:12345-0@lo:38:kworker.0 00010000:02000000:3.0:1782488232.772956:0:190296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488232.772984:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455a2d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300137216:12345-0@lo:38:kworker.0 Request processed in 41us (581us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488232.772990:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14016 00000100:00100000:3.0:1782488232.773000:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14017 00000100:00100000:3.0:1782488232.773004:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455a2fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300137344:12345-0@lo:38:kworker.0 00010000:02000000:3.0:1782488232.773009:0:190296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488232.773027:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455a2fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300137344:12345-0@lo:38:kworker.0 Request processed in 22us (580us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488232.773032:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14017 00000100:00100000:1.0:1782488232.787250:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300137088:0@lo:400:kworker.0 00000100:00080000:1.0:1782488232.787466:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488232.787472:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488232.787476:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300137216:0@lo:38:kworker.0 00000100:00080000:1.0:1782488232.787486:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488232.787490:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488232.787492:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300137344:0@lo:38:kworker.0 00010000:00010000:3.0:1782488233.606429:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:02000400:2.0:1782488234.306316:0:229334:0:(genops.c:1873:obd_exports_barrier()) lustre-MDT0000 is waiting for obd_unlinked_exports more than 16 seconds. The obd refcount = 5. Is it stuck? 00000020:02000400:2.0:1782488234.330521:0:229334:0:(genops.c:1840:print_export_data()) lustre-MDT0000: UNLINKED ffff9cb551430000 5c4ca15f-f34e-4111-b105-de67ffe3ec92 192.168.202.21@tcp 2 (0 0 0) 1 0 0 0: 0000000000000000 8589967277 stale:0 00010000:00010000:0.0:1782488234.627210:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488235.649737:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488236.682279:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488237.698721:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1782488237.895966:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1782488237.895973:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1782488237.895977:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1782488237.895985:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7860 00000100:00080000:3.0:1782488237.895989:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1782488237.896098:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488237.896253:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb455a2ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300137472:0@lo:400:kworker.0 00000100:00100000:1.0:1782488237.896324:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488237.896351:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300137472 00000100:00100000:0.0:1782488237.896358:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300137472 00000100:00100000:0.0:1782488237.896418:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 590 00000100:00100000:0.0:1782488237.896427:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f79d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300137472:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488237.896482:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f79d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300137472:12345-0@lo:400:kworker.0 Request processed in 56us (157us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488237.896489:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 590 00000100:00100000:0.0:1782488237.896533:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb455a2ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300137472:0@lo:400:kworker.0 00000100:00100000:3.0:1782488237.897227:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb455a2ed80 x1869066300137600/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1782488237.908194:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1782488237.908205:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1782488237.908209:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1782488237.908217:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7860 00000100:00080000:3.0:1782488237.908223:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1782488237.908230:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:3.0:1782488237.908280:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb455a2df80 x1869066300137728/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1782488237.909908:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb455a2ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300137600:0@lo:38:kworker.0 00000100:00100000:1.0:1782488237.909983:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488237.910121:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300137600 00000100:00100000:3.0:1782488237.910130:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300137600 00000100:00100000:3.0:1782488237.910173:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14018 00000100:00100000:3.0:1782488237.910180:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb450349500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300137600:12345-0@lo:38:kworker.0 00010000:02000000:3.0:1782488237.910197:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488237.912135:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb455a2df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300137728:0@lo:38:kworker.0 00000100:00100000:1.0:1782488237.912216:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488237.912294:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300137728 00000100:00100000:1.0:1782488237.912302:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300137728 00000100:00100000:1.0:1782488237.912335:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14019 00000100:00100000:1.0:1782488237.912341:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45034a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300137728:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488237.912357:0:190296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488237.912394:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45034a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300137728:12345-0@lo:38:kworker.0 Request processed in 52us (178us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488237.912401:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14019 00000100:00080000:1.0:1782488237.913181:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488237.913187:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488237.913191:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb455a2df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300137728:0@lo:38:kworker.0 00000100:00100000:3.0:1782488237.945561:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb450349500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300137600:12345-0@lo:38:kworker.0 Request processed in 35368us (35565us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488237.945578:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14018 00000100:00080000:3.0:1782488237.945673:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1782488237.945679:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:3.0:1782488237.945683:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb455a2ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300137600:0@lo:38:kworker.0 00010000:00010000:0.0:1782488238.725952:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488239.745724:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488240.773789:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488241.793862:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488242.820698:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1782488243.010547:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488243.010555:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488243.010558:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488243.010567:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7865 00000100:00080000:1.0:1782488243.010570:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488243.010577:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488243.010623:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb450348e00 x1869066300137984/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1782488243.010641:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488243.010645:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488243.010646:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488243.010649:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7865 00000100:00080000:1.0:1782488243.010652:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488243.010656:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488243.010667:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb45034ad80 x1869066300138112/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1782488243.010697:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45034bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300137856:0@lo:400:kworker.0 00000100:00100000:1.0:1782488243.010807:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488243.010882:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb450348e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300137984:0@lo:38:kworker.0 00000100:00100000:1.0:1782488243.010932:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488243.010960:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45034ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300138112:0@lo:38:kworker.0 00000100:00100000:1.0:1782488243.011018:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488243.011058:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300137856 00000100:00100000:1.0:1782488243.011065:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300137856 00000100:00100000:1.0:1782488243.011122:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 591 00000100:00100000:1.0:1782488243.011131:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f79d450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300137856:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488243.011180:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f79d450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300137856:12345-0@lo:400:kworker.0 Request processed in 49us (372us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488243.011188:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 591 00000100:00100000:2.0:1782488243.011231:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300137984 00000100:00100000:2.0:1782488243.011243:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300137984 00000100:00100000:1.0:1782488243.011249:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45034bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300137856:0@lo:400:kworker.0 00000100:00100000:2.0:1782488243.011264:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300138112 00000100:00100000:2.0:1782488243.011267:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300138112 00000100:00100000:2.0:1782488243.011293:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14020 00000100:00100000:2.0:1782488243.011309:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb450349f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300137984:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488243.011333:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488243.011372:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb450349f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300137984:12345-0@lo:38:kworker.0 Request processed in 62us (439us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488243.011440:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14021 00000100:00100000:1.0:1782488243.011446:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb450349c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300138112:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488243.011453:0:190296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488243.011483:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb450349c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300138112:12345-0@lo:38:kworker.0 Request processed in 36us (464us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488243.011488:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14021 00000100:00080000:1.0:1782488243.011566:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488243.011572:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488243.011574:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb450348e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300137984:0@lo:38:kworker.0 00000100:00080000:1.0:1782488243.011582:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488243.011585:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488243.011588:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45034ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300138112:0@lo:38:kworker.0 00000100:00100000:2.0:1782488243.011931:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14020 00010000:00010000:3.0:1782488243.848225:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488244.870015:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488245.905590:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488246.917595:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488247.941642:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488248.129758:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488248.129765:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488248.129769:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488248.129846:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7870 00000100:00080000:2.0:1782488248.129851:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488248.129858:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488248.129897:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff1f80 x1869066300138368/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488248.129915:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488248.129919:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488248.129921:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488248.129924:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7870 00000100:00080000:2.0:1782488248.129926:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488248.129930:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488248.129940:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff1c00 x1869066300138496/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488248.129969:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300138240:0@lo:400:kworker.0 00000100:00100000:2.0:1782488248.130316:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488248.130395:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300138368:0@lo:38:kworker.0 00000100:00100000:2.0:1782488248.130422:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488248.130467:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300138496:0@lo:38:kworker.0 00000100:00100000:2.0:1782488248.130489:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488248.130552:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300138368 00000100:00100000:2.0:1782488248.130559:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300138368 00000100:00100000:2.0:1782488248.130598:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300138496 00000100:00100000:2.0:1782488248.130601:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300138496 00000100:00100000:2.0:1782488248.130636:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14022 00000100:00100000:2.0:1782488248.130643:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300138368:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488248.130660:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488248.130711:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300138368:12345-0@lo:38:kworker.0 Request processed in 67us (287us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488248.130718:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14022 00000100:00100000:2.0:1782488248.130735:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14023 00000100:00100000:2.0:1782488248.130738:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300138496:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488248.130745:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488248.130764:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300138496:12345-0@lo:38:kworker.0 Request processed in 24us (274us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488248.130769:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14023 00000100:00080000:2.0:1782488248.132167:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488248.132177:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488248.132182:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300138368:0@lo:38:kworker.0 00000100:00080000:2.0:1782488248.132194:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488248.132198:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488248.132201:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300138496:0@lo:38:kworker.0 00000100:00100000:0.0:1782488248.134188:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300138240 00000100:00100000:0.0:1782488248.134200:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300138240 00000100:00100000:0.0:1782488248.134272:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 592 00000100:00100000:0.0:1782488248.134282:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f79f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300138240:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488248.134382:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f79f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300138240:12345-0@lo:400:kworker.0 Request processed in 101us (4067us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488248.134389:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 592 00000100:00100000:2.0:1782488248.134720:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300138240:0@lo:400:kworker.0 00010000:00010000:1.0:1782488248.973983:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488249.985988:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488251.009824:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488252.033937:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488253.068232:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488253.253365:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488253.253384:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488253.253388:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488253.253422:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7875 00000100:00080000:2.0:1782488253.253427:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488253.253434:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488253.253470:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff1c00 x1869066300138752/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488253.253488:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488253.253491:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488253.253493:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488253.253496:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7875 00000100:00080000:2.0:1782488253.253498:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488253.253502:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488253.253512:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff1f80 x1869066300138880/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488253.253538:0:3315:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1782488253.253575:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300138624:0@lo:400:kworker.0 00000100:00100000:2.0:1782488253.253694:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488253.253756:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300138752:0@lo:38:kworker.0 00000100:00100000:2.0:1782488253.253803:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488253.253831:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300138880:0@lo:38:kworker.0 00000100:00100000:2.0:1782488253.253870:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488253.253940:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300138752 00000100:00100000:2.0:1782488253.253948:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300138752 00000100:00100000:2.0:1782488253.254001:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300138880 00000100:00100000:2.0:1782488253.254004:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300138880 00000100:00100000:2.0:1782488253.254035:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14024 00000100:00100000:2.0:1782488253.254041:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300138752:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488253.254076:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488253.254121:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300138752:12345-0@lo:38:kworker.0 Request processed in 78us (317us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488253.254129:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14024 00000100:00100000:2.0:1782488253.254147:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14025 00000100:00100000:2.0:1782488253.254151:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300138880:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488253.254159:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488253.254184:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300138880:12345-0@lo:38:kworker.0 Request processed in 30us (313us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488253.254189:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14025 00000100:00080000:2.0:1782488253.254243:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488253.254249:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488253.254252:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300138752:0@lo:38:kworker.0 00000100:00080000:2.0:1782488253.254261:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488253.254265:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488253.254267:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300138880:0@lo:38:kworker.0 00000100:00100000:1.0:1782488253.254391:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300138624 00000100:00100000:1.0:1782488253.254397:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300138624 00000100:00100000:1.0:1782488253.254473:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 593 00000100:00100000:1.0:1782488253.254480:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f79fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300138624:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488253.254550:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f79fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300138624:12345-0@lo:400:kworker.0 Request processed in 70us (855us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488253.254559:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 593 00000100:00100000:1.0:1782488253.254625:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300138624:0@lo:400:kworker.0 00010000:00010000:3.0:1782488254.084462:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488255.106803:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488256.129813:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488257.157823:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488258.185003:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488258.369998:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488258.370006:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488258.370009:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488258.370019:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7880 00000100:00080000:2.0:1782488258.370023:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488258.370030:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488258.370055:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff1c00 x1869066300139136/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488258.370129:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488258.370134:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488258.370136:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488258.370142:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7880 00000100:00080000:2.0:1782488258.370145:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488258.370150:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488258.370163:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff0a80 x1869066300139264/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488258.370191:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300139008:0@lo:400:kworker.0 00000100:00100000:2.0:1782488258.371019:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488258.371088:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300139136:0@lo:38:kworker.0 00000100:00100000:2.0:1782488258.371116:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488258.371130:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300139264:0@lo:38:kworker.0 00000100:00100000:2.0:1782488258.371146:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488258.371170:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300139008 00000100:00100000:2.0:1782488258.371177:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300139008 00000100:00100000:2.0:1782488258.371624:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 594 00000100:00100000:2.0:1782488258.371636:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f79cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300139008:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488258.371687:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f79cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300139008:12345-0@lo:400:kworker.0 Request processed in 54us (670us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488258.371694:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 594 00000100:00100000:2.0:1782488258.371743:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300139136 00000100:00100000:2.0:1782488258.371747:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300139136 00000100:00100000:2.0:1782488258.371778:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300139264 00000100:00100000:2.0:1782488258.371791:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300139264 00000100:00100000:2.0:1782488258.371807:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14026 00000100:00100000:2.0:1782488258.371812:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300139136:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488258.371828:0:190296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488258.375095:0:213935:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14027 00000100:00100000:1.0:1782488258.375108:0:213935:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300139264:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488258.375131:0:213935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488258.375218:0:213935:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300139264:12345-0@lo:38:kworker.0 Request processed in 110us (4069us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488258.375226:0:213935:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14027 00000100:00080000:1.0:1782488258.375335:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488258.375342:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488258.375347:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300139264:0@lo:38:kworker.0 00000100:00100000:1.0:1782488258.377491:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300139008:0@lo:400:kworker.0 00000100:00100000:2.0:1782488258.381796:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300139136:12345-0@lo:38:kworker.0 Request processed in 9973us (10669us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488258.381811:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14026 00000100:00080000:2.0:1782488258.381873:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488258.381879:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488258.381883:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300139136:0@lo:38:kworker.0 00010000:00010000:1.0:1782488259.207563:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488260.225866:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488261.249889:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488262.278143:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488263.298976:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488263.491499:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488263.491507:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488263.491511:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488263.491519:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7885 00000100:00080000:2.0:1782488263.491523:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488263.491530:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488263.491552:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff0000 x1869066300139520/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488263.491569:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488263.491572:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488263.491574:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488263.491577:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7885 00000100:00080000:2.0:1782488263.491580:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488263.491583:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488263.491594:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff1f80 x1869066300139648/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488263.491692:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1782488263.491721:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300139392:0@lo:400:kworker.0 00000100:00100000:3.0:1782488263.491911:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300139520:0@lo:38:kworker.0 00000100:00100000:3.0:1782488263.491967:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488263.492026:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300139648:0@lo:38:kworker.0 00000100:00100000:3.0:1782488263.492049:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488263.492085:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300139520 00000100:00100000:3.0:1782488263.492092:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300139520 00000100:00100000:3.0:1782488263.492123:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300139648 00000100:00100000:3.0:1782488263.492126:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300139648 00000100:00100000:3.0:1782488263.492145:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14028 00000100:00100000:3.0:1782488263.492151:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455a2ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300139520:12345-0@lo:38:kworker.0 00010000:02000000:3.0:1782488263.492167:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488263.492222:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455a2ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300139520:12345-0@lo:38:kworker.0 Request processed in 69us (254us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488263.492228:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14028 00000100:00100000:3.0:1782488263.492245:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14029 00000100:00100000:3.0:1782488263.492249:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455a2dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300139648:12345-0@lo:38:kworker.0 00010000:02000000:3.0:1782488263.492255:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488263.492296:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455a2dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300139648:12345-0@lo:38:kworker.0 Request processed in 45us (246us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488263.492302:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14029 00000100:00080000:3.0:1782488263.492376:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1782488263.492381:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:3.0:1782488263.492384:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300139520:0@lo:38:kworker.0 00000100:00080000:3.0:1782488263.492394:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1782488263.492406:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:3.0:1782488263.492408:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300139648:0@lo:38:kworker.0 00000100:00100000:3.0:1782488263.493040:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488263.493639:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300139392 00000100:00100000:2.0:1782488263.493648:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300139392 00000100:00100000:2.0:1782488263.493880:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 595 00000100:00100000:2.0:1782488263.493891:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f79dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300139392:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488263.493964:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f79dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300139392:12345-0@lo:400:kworker.0 Request processed in 75us (925us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488263.493971:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 595 00000100:00100000:3.0:1782488263.494871:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300139392:0@lo:400:kworker.0 00010000:00010000:1.0:1782488264.328008:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488265.348247:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488266.378571:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:02000400:0.0:1782488266.567792:0:229334:0:(genops.c:1873:obd_exports_barrier()) lustre-MDT0000 is waiting for obd_unlinked_exports more than 32 seconds. The obd refcount = 5. Is it stuck? 00000020:02000400:0.0:1782488266.574980:0:229334:0:(genops.c:1840:print_export_data()) lustre-MDT0000: UNLINKED ffff9cb551430000 5c4ca15f-f34e-4111-b105-de67ffe3ec92 192.168.202.21@tcp 2 (0 0 0) 1 0 0 0: 0000000000000000 8589967277 stale:0 00010000:00010000:1.0:1782488267.394433:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488268.427982:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488268.610093:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488268.610100:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488268.610104:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488268.610112:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7890 00000100:00080000:2.0:1782488268.610117:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488268.610123:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488268.610144:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff2d80 x1869066300139904/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488268.610158:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488268.610161:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488268.610163:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488268.610166:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7890 00000100:00080000:2.0:1782488268.610168:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488268.610171:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488268.610181:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff0e00 x1869066300140032/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488268.611386:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1782488268.611423:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300139776:0@lo:400:kworker.0 00000100:00100000:2.0:1782488268.611504:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488268.611596:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300139776 00000100:00100000:2.0:1782488268.611603:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300139776 00000100:00100000:2.0:1782488268.611670:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 596 00000100:00100000:2.0:1782488268.611679:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f79c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300139776:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782488268.612006:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300139904:0@lo:38:kworker.0 00000100:00100000:3.0:1782488268.612043:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488268.612083:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300140032:0@lo:38:kworker.0 00000100:00100000:3.0:1782488268.612106:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488268.612180:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300139904 00000100:00100000:3.0:1782488268.612185:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300139904 00000100:00100000:3.0:1782488268.612229:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300140032 00000100:00100000:3.0:1782488268.612232:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300140032 00000100:00100000:3.0:1782488268.612249:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14030 00000100:00100000:3.0:1782488268.612255:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455a2ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300139904:12345-0@lo:38:kworker.0 00010000:02000000:3.0:1782488268.612280:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488268.612323:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455a2ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300139904:12345-0@lo:38:kworker.0 Request processed in 66us (277us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488268.612329:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14030 00000100:00100000:3.0:1782488268.612359:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14031 00000100:00100000:3.0:1782488268.612363:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455a2d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300140032:12345-0@lo:38:kworker.0 00010000:02000000:3.0:1782488268.612370:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488268.612393:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455a2d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300140032:12345-0@lo:38:kworker.0 Request processed in 28us (286us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488268.612398:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14031 00000100:00080000:3.0:1782488268.612785:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1782488268.612823:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:3.0:1782488268.612827:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300139904:0@lo:38:kworker.0 00000100:00080000:3.0:1782488268.612841:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1782488268.612845:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:3.0:1782488268.612847:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300140032:0@lo:38:kworker.0 00000100:00100000:2.0:1782488268.615408:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f79c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300139776:12345-0@lo:400:kworker.0 Request processed in 3718us (3893us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488268.615425:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 596 00000100:00100000:2.0:1782488268.615574:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300139776:0@lo:400:kworker.0 00010000:00010000:0.0:1782488269.441806:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488270.475349:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488271.493442:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488272.519301:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488273.538754:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1782488273.747471:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488273.747478:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488273.747481:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488273.747489:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7896 00000100:00080000:1.0:1782488273.747493:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488273.747499:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488273.747516:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45034ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300140160:0@lo:400:kworker.0 00000100:00100000:1.0:1782488273.747646:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488273.747751:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300140160 00000100:00100000:1.0:1782488273.747758:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300140160 00000100:00100000:1.0:1782488273.747814:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 597 00000100:00100000:1.0:1782488273.747823:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f79ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300140160:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488273.747886:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f79ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300140160:12345-0@lo:400:kworker.0 Request processed in 64us (239us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488273.747893:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 597 00000100:00100000:1.0:1782488273.747954:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45034ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300140160:0@lo:400:kworker.0 00000100:00100000:1.0:1782488273.748000:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb45034ad80 x1869066300140288/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1782488273.748017:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488273.748021:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488273.748023:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488273.748026:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7896 00000100:00080000:1.0:1782488273.748028:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488273.748031:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488273.748069:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45034ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300140288:0@lo:38:kworker.0 00000100:00100000:1.0:1782488273.748099:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488273.748129:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300140288 00000100:00100000:1.0:1782488273.748132:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300140288 00000100:00100000:1.0:1782488273.748154:0:213935:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14032 00000100:00100000:1.0:1782488273.748159:0:213935:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb450348e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300140288:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488273.748172:0:213935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488273.748203:0:213935:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb450348e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300140288:12345-0@lo:38:kworker.0 Request processed in 43us (103us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488273.748209:0:213935:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14032 00000100:00080000:1.0:1782488273.748249:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488273.748254:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488273.748257:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45034ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300140288:0@lo:38:kworker.0 00000100:00100000:1.0:1782488273.748281:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb45034ad80 x1869066300140416/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1782488273.748300:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45034ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300140416:0@lo:38:kworker.0 00000100:00100000:1.0:1782488273.748320:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488273.748344:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300140416 00000100:00100000:1.0:1782488273.748347:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300140416 00000100:00100000:1.0:1782488273.748366:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14033 00000100:00100000:1.0:1782488273.748370:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45034bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300140416:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488273.748377:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488273.748405:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45034bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300140416:12345-0@lo:38:kworker.0 Request processed in 33us (83us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488273.748411:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14033 00000100:00080000:1.0:1782488273.748435:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488273.748445:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488273.748448:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45034ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300140416:0@lo:38:kworker.0 00010000:00010000:0.0:1782488274.569915:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488275.587731:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488276.616325:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488277.633787:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488278.658732:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488278.860253:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488278.860268:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488278.860272:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488278.860282:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7901 00000100:00080000:0.0:1782488278.860288:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488278.860296:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488278.860343:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb579577480 x1869066300140672/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1782488278.860361:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488278.860365:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488278.860367:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488278.860370:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7901 00000100:00080000:0.0:1782488278.860372:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488278.860376:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488278.860386:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb579575f80 x1869066300140800/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488278.860411:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1782488278.860429:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579574000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300140544:0@lo:400:kworker.0 00000100:00100000:0.0:1782488278.860750:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488278.860820:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579577480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300140672:0@lo:38:kworker.0 00000100:00100000:0.0:1782488278.860852:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488278.860872:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579575f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300140800:0@lo:38:kworker.0 00000100:00100000:0.0:1782488278.860890:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488278.860938:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300140544 00000100:00100000:0.0:1782488278.860946:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300140544 00000100:00100000:0.0:1782488278.861069:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 598 00000100:00100000:0.0:1782488278.861079:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f79e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300140544:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488278.861132:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f79e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300140544:12345-0@lo:400:kworker.0 Request processed in 53us (382us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488278.861140:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 598 00000100:00100000:0.0:1782488278.861176:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300140672 00000100:00100000:0.0:1782488278.861200:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300140672 00000100:00100000:0.0:1782488278.861219:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300140800 00000100:00100000:0.0:1782488278.861221:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300140800 00000100:00100000:0.0:1782488278.861236:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14034 00000100:00100000:0.0:1782488278.861241:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579575c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300140672:12345-0@lo:38:kworker.0 00000100:00100000:1.0:1782488278.861394:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579574000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300140544:0@lo:400:kworker.0 00000100:00100000:1.0:1782488278.861424:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14035 00000100:00100000:1.0:1782488278.861429:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579576a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300140800:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488278.861457:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488278.861491:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579575c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300140672:12345-0@lo:38:kworker.0 Request processed in 248us (638us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488278.861508:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14034 00000100:00080000:0.0:1782488278.861544:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488278.861549:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488278.861553:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579577480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300140672:0@lo:38:kworker.0 00010000:02000000:1.0:1782488278.867391:0:190296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488278.870524:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579576a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300140800:12345-0@lo:38:kworker.0 Request processed in 9083us (9622us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488278.870542:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14035 00000100:00080000:1.0:1782488278.870629:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488278.870638:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488278.870644:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579575f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300140800:0@lo:38:kworker.0 00010000:00010000:0.0:1782488279.704546:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488280.705838:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488281.736649:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488282.758844:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488283.777840:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488283.974434:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488283.974442:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488283.974446:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488283.974454:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7906 00000100:00080000:0.0:1782488283.974458:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488283.974465:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488283.974485:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb579575180 x1869066300141056/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488283.974686:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579577480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300140928:0@lo:400:kworker.0 00000100:00100000:2.0:1782488283.974774:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488283.974845:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300140928 00000100:00100000:2.0:1782488283.974852:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300140928 00000100:00100000:2.0:1782488283.974897:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 599 00000100:00100000:2.0:1782488283.974906:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f79e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300140928:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488283.974958:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f79e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300140928:12345-0@lo:400:kworker.0 Request processed in 53us (184us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488283.974966:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 599 00000100:00100000:2.0:1782488283.975017:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579577480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300140928:0@lo:400:kworker.0 00000100:00080000:0.0:1782488283.975686:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488283.975726:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488283.975737:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488283.975742:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7906 00000100:00080000:0.0:1782488283.975745:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488283.975749:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488283.975768:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb579575880 x1869066300141184/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488283.975800:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579575180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300141056:0@lo:38:kworker.0 00000100:00100000:0.0:1782488283.975835:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488283.975858:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579575880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300141184:0@lo:38:kworker.0 00000100:00100000:0.0:1782488283.975879:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488283.975916:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300141056 00000100:00100000:0.0:1782488283.975921:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300141056 00000100:00100000:0.0:1782488283.975943:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300141184 00000100:00100000:0.0:1782488283.975946:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300141184 00000100:00100000:0.0:1782488283.975961:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14036 00000100:00100000:0.0:1782488283.975966:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579577480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300141056:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488283.975982:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488283.976034:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579577480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300141056:12345-0@lo:38:kworker.0 Request processed in 66us (198us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488283.976040:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14036 00000100:00100000:0.0:1782488283.976055:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14037 00000100:00100000:0.0:1782488283.976059:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579575f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300141184:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488283.976066:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488283.976090:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579575f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300141184:12345-0@lo:38:kworker.0 Request processed in 28us (209us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488283.976106:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14037 00000100:00080000:0.0:1782488283.976151:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488283.976157:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488283.976160:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579575180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300141056:0@lo:38:kworker.0 00000100:00080000:0.0:1782488283.976169:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488283.976172:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488283.976176:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579575880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300141184:0@lo:38:kworker.0 00010000:00010000:1.0:1782488284.802395:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488285.829635:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488286.854531:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488287.875512:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488288.898827:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488289.090133:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488289.090140:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488289.090144:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488289.090152:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7911 00000100:00080000:0.0:1782488289.090157:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488289.090164:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488289.090186:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb579575180 x1869066300141440/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1782488289.090202:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488289.090206:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488289.090208:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488289.090211:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7911 00000100:00080000:0.0:1782488289.090213:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488289.090217:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488289.090228:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb579574000 x1869066300141568/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488289.090302:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579575180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300141440:0@lo:38:kworker.0 00000100:00100000:0.0:1782488289.090370:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488289.090426:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579574000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300141568:0@lo:38:kworker.0 00000100:00100000:0.0:1782488289.090448:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488289.090499:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300141440 00000100:00100000:0.0:1782488289.090507:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300141440 00000100:00100000:0.0:1782488289.090534:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300141568 00000100:00100000:0.0:1782488289.090537:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300141568 00000100:00100000:0.0:1782488289.090554:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14038 00000100:00100000:0.0:1782488289.090560:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579575500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300141440:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488289.090576:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488289.090637:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579575500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300141440:12345-0@lo:38:kworker.0 Request processed in 76us (267us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488289.090644:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14038 00000100:00100000:0.0:1782488289.090662:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14039 00000100:00100000:0.0:1782488289.090666:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579576300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300141568:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488289.090674:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488289.090696:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579576300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300141568:12345-0@lo:38:kworker.0 Request processed in 28us (247us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488289.090703:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14039 00000100:00080000:0.0:1782488289.090742:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488289.090747:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488289.090751:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579575180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300141440:0@lo:38:kworker.0 00000100:00080000:0.0:1782488289.090760:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488289.090763:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488289.090772:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579574000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300141568:0@lo:38:kworker.0 00000100:00100000:2.0:1782488289.090913:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579575880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300141312:0@lo:400:kworker.0 00000100:00100000:2.0:1782488289.090951:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488289.091045:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300141312 00000100:00100000:0.0:1782488289.091049:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300141312 00000100:00100000:0.0:1782488289.091143:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 600 00000100:00100000:0.0:1782488289.091151:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455218050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300141312:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488289.091192:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455218050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300141312:12345-0@lo:400:kworker.0 Request processed in 43us (240us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488289.091199:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 600 00000100:00100000:2.0:1782488289.101510:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579575880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300141312:0@lo:400:kworker.0 00010000:00010000:1.0:1782488289.938991:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488290.952998:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488291.971079:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488292.993837:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488294.018851:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1782488294.221526:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488294.221532:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488294.221536:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488294.221544:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7916 00000100:00080000:1.0:1782488294.221547:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488294.221554:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488294.221575:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb45034a680 x1869066300141824/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1782488294.221591:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488294.221595:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488294.221620:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488294.221624:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7916 00000100:00080000:1.0:1782488294.221626:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488294.221630:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488294.221640:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb450348700 x1869066300141952/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1782488294.221676:0:3315:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1782488294.221692:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45034ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300141696:0@lo:400:kworker.0 00000100:00100000:1.0:1782488294.221769:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488294.221818:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45034a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300141824:0@lo:38:kworker.0 00000100:00100000:1.0:1782488294.221842:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488294.221856:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb450348700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300141952:0@lo:38:kworker.0 00000100:00100000:1.0:1782488294.221911:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488294.221939:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300141696 00000100:00100000:0.0:1782488294.222162:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300141824 00000100:00100000:0.0:1782488294.222170:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300141824 00000100:00100000:0.0:1782488294.222199:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300141952 00000100:00100000:0.0:1782488294.222201:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300141952 00000100:00100000:0.0:1782488294.222215:0:213935:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14040 00000100:00100000:0.0:1782488294.222222:0:213935:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb450348000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300141824:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488294.222239:0:213935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488294.223475:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300141696 00000100:00100000:1.0:1782488294.223539:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 601 00000100:00100000:1.0:1782488294.223548:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45521a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300141696:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488294.223635:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45521a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300141696:12345-0@lo:400:kworker.0 Request processed in 89us (1865us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488294.223654:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 601 00000100:00100000:1.0:1782488294.223698:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14041 00000100:00100000:1.0:1782488294.223704:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45034b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300141952:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488294.223717:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488294.223854:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45034b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300141952:12345-0@lo:38:kworker.0 Request processed in 149us (1948us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488294.223861:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14041 00000100:00080000:1.0:1782488294.225114:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488294.225122:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488294.225127:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb450348700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300141952:0@lo:38:kworker.0 00000100:00100000:1.0:1782488294.225164:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45034ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300141696:0@lo:400:kworker.0 00000100:00100000:0.0:1782488294.239189:0:213935:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb450348000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300141824:12345-0@lo:38:kworker.0 Request processed in 16958us (17337us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488294.239205:0:213935:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14040 00000100:00080000:0.0:1782488294.239274:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488294.239280:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488294.239284:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45034a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300141824:0@lo:38:kworker.0 00010000:00010000:1.0:1782488295.042265:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488296.067866:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488297.090803:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488298.115358:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488299.140292:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1782488299.342876:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1782488299.342884:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1782488299.342888:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1782488299.342906:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7921 00000100:00080000:3.0:1782488299.342912:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1782488299.342919:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:3.0:1782488299.342944:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb455a2df80 x1869066300142208/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1782488299.342961:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1782488299.342964:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1782488299.342965:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1782488299.342981:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7921 00000100:00080000:3.0:1782488299.342983:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1782488299.342987:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:3.0:1782488299.342999:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb455a2ce00 x1869066300142336/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1782488299.343038:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1782488299.343056:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb455a2d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300142080:0@lo:400:kworker.0 00000100:00100000:3.0:1782488299.343124:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488299.343177:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb455a2df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300142208:0@lo:38:kworker.0 00000100:00100000:3.0:1782488299.343205:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488299.343233:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb455a2ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300142336:0@lo:38:kworker.0 00000100:00100000:3.0:1782488299.343254:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488299.343293:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300142080 00000100:00100000:3.0:1782488299.343300:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300142080 00000100:00100000:0.0:1782488299.343338:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300142208 00000100:00100000:0.0:1782488299.343344:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300142208 00000100:00100000:3.0:1782488299.343348:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 602 00000100:00100000:3.0:1782488299.343359:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455219050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300142080:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488299.343372:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300142336 00000100:00100000:0.0:1782488299.343376:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300142336 00000100:00100000:0.0:1782488299.343396:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14042 00000100:00100000:0.0:1782488299.343402:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455a2c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300142208:12345-0@lo:38:kworker.0 00000100:00100000:3.0:1782488299.343412:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455219050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300142080:12345-0@lo:400:kworker.0 Request processed in 54us (286us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488299.343419:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 602 00000100:00100000:3.0:1782488299.343452:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14043 00010000:02000000:0.0:1782488299.343456:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488299.343461:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455a2f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300142336:12345-0@lo:38:kworker.0 00010000:02000000:3.0:1782488299.343470:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488299.343509:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455a2c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300142208:12345-0@lo:38:kworker.0 Request processed in 105us (301us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488299.343510:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455a2f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300142336:12345-0@lo:38:kworker.0 Request processed in 49us (254us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488299.343525:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14042 00000100:00100000:0.0:1782488299.343880:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb455a2d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300142080:0@lo:400:kworker.0 00000100:00080000:0.0:1782488299.349109:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488299.349123:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488299.349129:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb455a2df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300142208:0@lo:38:kworker.0 00000100:00080000:0.0:1782488299.349150:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488299.349155:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488299.349159:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb455a2ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300142336:0@lo:38:kworker.0 00000100:00100000:3.0:1782488299.350406:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14043 00010000:00010000:2.0:1782488300.161717:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488301.185746:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488302.211178:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488303.233767:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488304.265011:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1782488304.450647:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488304.450655:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488304.450660:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488304.450670:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7926 00000100:00080000:1.0:1782488304.450675:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488304.450682:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488304.450710:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb450348700 x1869066300142592/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1782488304.450729:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488304.450732:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488304.450734:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488304.450738:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7926 00000100:00080000:1.0:1782488304.450741:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488304.450745:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488304.450760:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb45034b480 x1869066300142720/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1782488304.450795:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45034ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300142464:0@lo:400:kworker.0 00000100:00100000:1.0:1782488304.450875:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488304.450988:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300142464 00000100:00100000:1.0:1782488304.450997:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300142464 00000100:00100000:1.0:1782488304.451051:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 603 00000100:00100000:1.0:1782488304.451062:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455218850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300142464:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488304.451118:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455218850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300142464:12345-0@lo:400:kworker.0 Request processed in 57us (245us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488304.451127:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 603 00000100:00100000:1.0:1782488304.451217:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45034ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300142464:0@lo:400:kworker.0 00000100:00100000:1.0:1782488304.451261:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb450348700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300142592:0@lo:38:kworker.0 00000100:00100000:1.0:1782488304.451296:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488304.451322:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45034b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300142720:0@lo:38:kworker.0 00000100:00100000:1.0:1782488304.451349:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488304.451392:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300142592 00000100:00100000:1.0:1782488304.451397:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300142592 00000100:00100000:1.0:1782488304.451422:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300142720 00000100:00100000:1.0:1782488304.451425:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300142720 00000100:00100000:1.0:1782488304.451448:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14044 00000100:00100000:1.0:1782488304.451525:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45034ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300142592:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488304.451541:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488304.451584:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45034ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300142592:12345-0@lo:38:kworker.0 Request processed in 127us (289us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488304.451591:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14044 00000100:00100000:1.0:1782488304.451632:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14045 00000100:00100000:1.0:1782488304.451638:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45034a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300142720:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488304.451646:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488304.451677:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45034a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300142720:12345-0@lo:38:kworker.0 Request processed in 38us (328us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488304.451693:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14045 00000100:00080000:1.0:1782488304.451747:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488304.451753:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488304.451757:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb450348700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300142592:0@lo:38:kworker.0 00000100:00080000:1.0:1782488304.451771:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488304.451776:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488304.451780:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45034b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300142720:0@lo:38:kworker.0 00010000:00010000:0.0:1782488305.291495:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488306.305902:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488307.331378:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488308.364342:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488309.386162:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1782488309.571571:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488309.571577:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488309.571581:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488309.571588:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7931 00000100:00080000:1.0:1782488309.571592:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488309.571598:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488309.571634:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb450348700 x1869066300142976/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1782488309.571658:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488309.571662:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488309.571663:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488309.571666:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7931 00000100:00080000:1.0:1782488309.571668:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488309.571672:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488309.571684:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb45013b800 x1869066300143104/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1782488309.571975:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45034b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300142848:0@lo:400:kworker.0 00000100:00100000:1.0:1782488309.572063:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488309.572134:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300142848 00000100:00100000:1.0:1782488309.572142:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300142848 00000100:00100000:1.0:1782488309.572183:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 604 00000100:00100000:1.0:1782488309.572191:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45521ac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300142848:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488309.572240:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45521ac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300142848:12345-0@lo:400:kworker.0 Request processed in 49us (177us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488309.572247:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 604 00000100:00100000:1.0:1782488309.572305:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45034b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300142848:0@lo:400:kworker.0 00000100:00100000:2.0:1782488309.572539:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb450348700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300142976:0@lo:38:kworker.0 00000100:00100000:2.0:1782488309.572572:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488309.572598:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45013b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300143104:0@lo:38:kworker.0 00000100:00100000:2.0:1782488309.572638:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488309.572710:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300142976 00000100:00100000:2.0:1782488309.572714:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300142976 00000100:00100000:2.0:1782488309.572735:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300143104 00000100:00100000:2.0:1782488309.572738:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300143104 00000100:00100000:2.0:1782488309.572753:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14046 00000100:00100000:2.0:1782488309.572758:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300142976:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488309.572773:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488309.572790:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14047 00000100:00100000:1.0:1782488309.572795:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300143104:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488309.572803:0:190296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488309.572805:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300142976:12345-0@lo:38:kworker.0 Request processed in 47us (233us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488309.572812:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14046 00000100:00100000:1.0:1782488309.572826:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300143104:12345-0@lo:38:kworker.0 Request processed in 29us (188us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488309.572831:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14047 00000100:00080000:2.0:1782488309.572843:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488309.572848:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488309.572850:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb450348700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300142976:0@lo:38:kworker.0 00000100:00080000:2.0:1782488309.572859:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488309.572862:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488309.572865:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45013b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300143104:0@lo:38:kworker.0 00010000:00010000:3.0:1782488310.401955:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488311.425745:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488312.449910:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488313.473775:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488314.506851:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488314.701640:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488314.701647:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488314.701677:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488314.701685:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7936 00000100:00080000:2.0:1782488314.701690:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488314.701697:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488314.701708:0:3315:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1782488314.701724:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300143232:0@lo:400:kworker.0 00000100:00100000:2.0:1782488314.701853:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488314.701933:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300143232 00000100:00100000:2.0:1782488314.701941:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300143232 00000100:00100000:2.0:1782488314.702015:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 605 00000100:00100000:2.0:1782488314.702025:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455219850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300143232:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488314.702076:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455219850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300143232:12345-0@lo:400:kworker.0 Request processed in 53us (223us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488314.702084:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 605 00000100:00100000:2.0:1782488314.702130:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300143232:0@lo:400:kworker.0 00000100:00100000:2.0:1782488314.702164:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff2300 x1869066300143360/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488314.702180:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488314.702184:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488314.702186:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488314.702189:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7936 00000100:00080000:2.0:1782488314.702191:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488314.702195:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488314.702202:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300143360:0@lo:38:kworker.0 00000100:00100000:2.0:1782488314.702252:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488314.702313:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300143360 00000100:00100000:2.0:1782488314.702317:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300143360 00000100:00100000:2.0:1782488314.702362:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14048 00000100:00100000:2.0:1782488314.702506:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb574b00050 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300143360:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488314.702532:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488314.702564:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb574b00050 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300143360:12345-0@lo:38:kworker.0 Request processed in 58us (312us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488314.712533:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb579574000 x1869066300143488/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488314.712681:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14048 00000100:00100000:2.0:1782488314.716054:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579574000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300143488:0@lo:38:kworker.0 00000100:00100000:2.0:1782488314.716132:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1782488314.716209:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488314.716215:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488314.716224:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300143360:0@lo:38:kworker.0 00000100:00100000:0.0:1782488314.717517:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300143488 00000100:00100000:0.0:1782488314.717527:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300143488 00000100:00100000:0.0:1782488314.717573:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14049 00000100:00100000:0.0:1782488314.717579:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300143488:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488314.717596:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488314.717703:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300143488:12345-0@lo:38:kworker.0 Request processed in 122us (1570us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488314.717710:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14049 00000100:00080000:2.0:1782488314.717800:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488314.717806:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488314.717809:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579574000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300143488:0@lo:38:kworker.0 00010000:00010000:2.0:1782488315.523497:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488316.546696:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488317.578766:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488318.593772:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488319.617707:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1782488319.816998:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488319.817005:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488319.817008:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488319.817015:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7942 00000100:00080000:1.0:1782488319.817020:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488319.817027:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488319.817049:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb450138380 x1869066300143744/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1782488319.817066:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488319.817070:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488319.817118:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488319.817123:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7942 00000100:00080000:1.0:1782488319.817125:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488319.817130:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488319.817143:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb450138700 x1869066300143872/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1782488319.817169:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1782488319.817198:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb450138a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300143616:0@lo:400:kworker.0 00000100:00100000:1.0:1782488319.817927:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488319.818004:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb450138380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300143744:0@lo:38:kworker.0 00000100:00100000:1.0:1782488319.818038:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488319.818060:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb450138700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300143872:0@lo:38:kworker.0 00000100:00100000:1.0:1782488319.818080:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488319.818136:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300143616 00000100:00100000:1.0:1782488319.818137:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300143744 00000100:00100000:1.0:1782488319.818144:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300143744 00000100:00100000:3.0:1782488319.818144:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300143616 00000100:00100000:1.0:1782488319.818181:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300143872 00000100:00100000:1.0:1782488319.818184:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300143872 00000100:00100000:3.0:1782488319.818185:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 606 00000100:00100000:3.0:1782488319.818192:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45521b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300143616:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488319.818199:0:213935:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14050 00000100:00100000:1.0:1782488319.818204:0:213935:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45013a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300143744:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488319.818220:0:213935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488319.818253:0:213935:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45013a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300143744:12345-0@lo:38:kworker.0 Request processed in 47us (213us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488319.818253:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45521b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300143616:12345-0@lo:400:kworker.0 Request processed in 60us (968us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488319.818260:0:213935:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14050 00000100:00100000:3.0:1782488319.818260:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 606 00000100:00100000:3.0:1782488319.818326:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14051 00000100:00100000:3.0:1782488319.818330:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb450139180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300143872:12345-0@lo:38:kworker.0 00010000:02000000:3.0:1782488319.818338:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488319.818363:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb450139180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300143872:12345-0@lo:38:kworker.0 Request processed in 31us (282us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488319.818369:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14051 00000100:00080000:1.0:1782488319.820473:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488319.820485:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488319.820489:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb450138380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300143744:0@lo:38:kworker.0 00000100:00080000:1.0:1782488319.820520:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488319.820524:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488319.820527:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb450138700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300143872:0@lo:38:kworker.0 00000100:00100000:1.0:1782488319.820567:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb450138a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300143616:0@lo:400:kworker.0 00010000:00010000:2.0:1782488320.641831:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488321.672624:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488322.693699:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488323.719889:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488324.738888:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1782488324.929761:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488324.929769:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488324.929773:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488324.929793:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7947 00000100:00080000:1.0:1782488324.929798:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488324.929805:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488324.929827:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb450138700 x1869066300144128/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1782488324.929844:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488324.929848:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488324.929849:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488324.929853:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7947 00000100:00080000:1.0:1782488324.929855:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488324.929858:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488324.929869:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb450138380 x1869066300144256/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1782488324.929893:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1782488324.929909:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb450138a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300144000:0@lo:400:kworker.0 00000100:00100000:1.0:1782488324.929995:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488324.930067:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb450138700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300144128:0@lo:38:kworker.0 00000100:00100000:1.0:1782488324.930093:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488324.930112:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb450138380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300144256:0@lo:38:kworker.0 00000100:00100000:1.0:1782488324.930129:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488324.930159:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300144000 00000100:00100000:1.0:1782488324.930165:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300144000 00000100:00100000:1.0:1782488324.930208:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 607 00000100:00100000:1.0:1782488324.930216:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45521b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300144000:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488324.930262:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45521b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300144000:12345-0@lo:400:kworker.0 Request processed in 46us (265us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488324.930268:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 607 00000100:00100000:1.0:1782488324.930301:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300144128 00000100:00100000:1.0:1782488324.930304:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300144128 00000100:00100000:1.0:1782488324.930320:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300144256 00000100:00100000:1.0:1782488324.930322:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300144256 00000100:00100000:1.0:1782488324.930336:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14052 00000100:00100000:1.0:1782488324.930340:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb450138e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300144128:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488324.930354:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488324.930384:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb450138e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300144128:12345-0@lo:38:kworker.0 Request processed in 43us (289us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488324.930390:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14052 00000100:00100000:1.0:1782488324.930401:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14053 00000100:00100000:1.0:1782488324.930406:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb450139c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300144256:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488324.930412:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488324.930435:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb450139c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300144256:12345-0@lo:38:kworker.0 Request processed in 28us (305us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488324.930440:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14053 00000100:00080000:1.0:1782488324.930485:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488324.930490:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488324.930493:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb450138700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300144128:0@lo:38:kworker.0 00000100:00080000:1.0:1782488324.930500:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488324.930503:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488324.930506:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb450138380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300144256:0@lo:38:kworker.0 00000100:00100000:1.0:1782488324.930535:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb450138a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300144000:0@lo:400:kworker.0 00010000:00010000:3.0:1782488325.775380:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488326.786294:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488327.810732:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488328.847041:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488329.857882:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488330.054347:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488330.054354:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488330.054358:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488330.054366:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7952 00000100:00080000:0.0:1782488330.054370:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488330.054376:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488330.054398:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb579574700 x1869066300144512/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1782488330.054414:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488330.054417:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488330.054419:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488330.054422:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7952 00000100:00080000:0.0:1782488330.054425:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488330.054429:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488330.054456:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb579576680 x1869066300144640/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1782488330.054456:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579575180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300144384:0@lo:400:kworker.0 00000100:00100000:0.0:1782488330.054495:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579574700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300144512:0@lo:38:kworker.0 00000100:00100000:1.0:1782488330.054538:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488330.058804:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488330.058922:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579576680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300144640:0@lo:38:kworker.0 00000100:00100000:0.0:1782488330.058963:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488330.059029:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300144512 00000100:00100000:0.0:1782488330.059037:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300144512 00000100:00100000:0.0:1782488330.059066:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300144640 00000100:00100000:0.0:1782488330.059069:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300144640 00000100:00100000:0.0:1782488330.059084:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14054 00000100:00100000:0.0:1782488330.059090:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579577100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300144512:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488330.059581:0:190296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488330.059665:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579577100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300144512:12345-0@lo:38:kworker.0 Request processed in 572us (868us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488330.059674:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14054 00000100:00100000:0.0:1782488330.059697:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14055 00000100:00100000:0.0:1782488330.059701:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579577800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300144640:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488330.059710:0:190296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488330.059747:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579577800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300144640:12345-0@lo:38:kworker.0 Request processed in 32us (770us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488330.059754:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14055 00000100:00080000:0.0:1782488330.059797:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488330.059802:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488330.059805:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579574700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300144512:0@lo:38:kworker.0 00000100:00080000:0.0:1782488330.059815:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488330.059818:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488330.059821:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579576680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300144640:0@lo:38:kworker.0 00000100:00100000:0.0:1782488330.059863:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300144384 00000100:00100000:0.0:1782488330.059880:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300144384 00000100:00100000:0.0:1782488330.059913:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 608 00000100:00100000:0.0:1782488330.059920:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45521bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300144384:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488330.059958:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45521bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300144384:12345-0@lo:400:kworker.0 Request processed in 39us (5421us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488330.059964:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 608 00000100:00100000:1.0:1782488330.064537:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579575180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300144384:0@lo:400:kworker.0 00010000:00010000:0.0:1782488330.886065:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488331.914435:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:02000400:0.0:1782488332.097684:0:229334:0:(genops.c:1873:obd_exports_barrier()) lustre-MDT0000 is waiting for obd_unlinked_exports more than 64 seconds. The obd refcount = 5. Is it stuck? 00000020:02000400:0.0:1782488332.104803:0:229334:0:(genops.c:1840:print_export_data()) lustre-MDT0000: UNLINKED ffff9cb551430000 5c4ca15f-f34e-4111-b105-de67ffe3ec92 192.168.202.21@tcp 2 (0 0 0) 1 0 0 0: 0000000000000000 8589967277 stale:0 00010000:00010000:0.0:1782488332.933291:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488333.955266:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488334.979768:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488335.170988:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488335.170995:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488335.170999:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488335.171007:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7957 00000100:00080000:2.0:1782488335.171011:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488335.171017:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488335.171051:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff1500 x1869066300144896/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488335.171067:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488335.171070:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488335.171072:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488335.171076:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7957 00000100:00080000:2.0:1782488335.171078:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488335.171082:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488335.171092:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff0700 x1869066300145024/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488335.171116:0:3315:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1782488335.171134:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300144768:0@lo:400:kworker.0 00000100:00100000:2.0:1782488335.171209:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488335.171261:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300144896:0@lo:38:kworker.0 00000100:00100000:2.0:1782488335.171290:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488335.171309:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300145024:0@lo:38:kworker.0 00000100:00100000:2.0:1782488335.171331:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488335.171364:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300144768 00000100:00100000:2.0:1782488335.171370:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300144768 00000100:00100000:2.0:1782488335.171410:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 609 00000100:00100000:2.0:1782488335.171417:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45521a050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300144768:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488335.171467:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45521a050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300144768:12345-0@lo:400:kworker.0 Request processed in 49us (257us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488335.171474:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 609 00000100:00100000:2.0:1782488335.171542:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300144896 00000100:00100000:2.0:1782488335.171552:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300144896 00000100:00100000:2.0:1782488335.171572:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300145024 00000100:00100000:2.0:1782488335.171574:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300145024 00000100:00100000:2.0:1782488335.171587:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14056 00000100:00100000:2.0:1782488335.171592:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300144896:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488335.171623:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488335.171654:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300144896:12345-0@lo:38:kworker.0 Request processed in 61us (364us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488335.171660:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14056 00000100:00100000:2.0:1782488335.171670:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14057 00000100:00100000:2.0:1782488335.171673:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300145024:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488335.171679:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488335.171712:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300145024:12345-0@lo:38:kworker.0 Request processed in 37us (381us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488335.171717:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14057 00000100:00100000:2.0:1782488335.171750:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300144768:0@lo:400:kworker.0 00000100:00080000:2.0:1782488335.171780:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488335.171784:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488335.171786:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300144896:0@lo:38:kworker.0 00000100:00080000:2.0:1782488335.171793:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488335.171795:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488335.171797:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300145024:0@lo:38:kworker.0 00010000:00010000:1.0:1782488336.001803:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488337.030892:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488338.049855:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488339.079527:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488340.097813:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1782488340.293390:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488340.293397:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488340.293402:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488340.293412:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7962 00000100:00080000:1.0:1782488340.293417:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488340.293425:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488340.293450:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb450138380 x1869066300145280/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1782488340.293469:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488340.293473:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488340.293476:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488340.293497:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7962 00000100:00080000:1.0:1782488340.293500:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488340.293505:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488340.293521:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb450138700 x1869066300145408/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1782488340.293553:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb450138a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300145152:0@lo:400:kworker.0 00000100:00100000:1.0:1782488340.293661:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488340.293749:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb450138380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300145280:0@lo:38:kworker.0 00000100:00100000:1.0:1782488340.293784:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488340.293811:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb450138700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300145408:0@lo:38:kworker.0 00000100:00100000:1.0:1782488340.293845:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488340.293888:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300145152 00000100:00100000:1.0:1782488340.293896:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300145152 00000100:00100000:1.0:1782488340.293951:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 610 00000100:00100000:1.0:1782488340.293962:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455219450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300145152:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488340.294021:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455219450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300145152:12345-0@lo:400:kworker.0 Request processed in 61us (361us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488340.294030:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 610 00000100:00100000:1.0:1782488340.294075:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300145280 00000100:00100000:1.0:1782488340.294079:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300145280 00000100:00100000:1.0:1782488340.294104:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300145408 00000100:00100000:1.0:1782488340.294178:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300145408 00000100:00100000:1.0:1782488340.294206:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14058 00000100:00100000:1.0:1782488340.294212:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb450138000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300145280:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488340.294228:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488340.294265:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb450138000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300145280:12345-0@lo:38:kworker.0 Request processed in 51us (481us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488340.294279:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14058 00000100:00100000:1.0:1782488340.294293:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14059 00000100:00100000:1.0:1782488340.294297:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb450139f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300145408:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488340.294305:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488340.294332:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb450139f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300145408:12345-0@lo:38:kworker.0 Request processed in 33us (486us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488340.294338:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14059 00000100:00080000:1.0:1782488340.294383:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488340.294388:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488340.294392:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb450138380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300145280:0@lo:38:kworker.0 00000100:00080000:1.0:1782488340.294401:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488340.294404:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488340.294407:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb450138700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300145408:0@lo:38:kworker.0 00000100:00100000:1.0:1782488340.294434:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb450138a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300145152:0@lo:400:kworker.0 00010000:00010000:0.0:1782488341.124014:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488342.147910:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488343.173185:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488344.197743:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488345.224009:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488345.414226:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488345.414234:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488345.414238:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488345.414245:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7967 00000100:00080000:0.0:1782488345.414250:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488345.414256:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488345.414277:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb579574700 x1869066300145664/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1782488345.414294:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488345.414298:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488345.414314:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488345.414318:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7967 00000100:00080000:0.0:1782488345.414320:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488345.414324:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488345.414336:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb579577b80 x1869066300145792/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488345.414365:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579576680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300145536:0@lo:400:kworker.0 00000100:00100000:0.0:1782488345.414436:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488345.414466:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579574700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300145664:0@lo:38:kworker.0 00000100:00100000:2.0:1782488345.414499:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488345.414514:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300145536 00000100:00100000:2.0:1782488345.414517:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579577b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300145792:0@lo:38:kworker.0 00000100:00100000:0.0:1782488345.414521:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300145536 00000100:00100000:2.0:1782488345.414545:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488345.414560:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 611 00000100:00100000:0.0:1782488345.414569:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455218c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300145536:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488345.414571:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300145664 00000100:00100000:2.0:1782488345.414574:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300145664 00000100:00100000:2.0:1782488345.414593:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300145792 00000100:00100000:2.0:1782488345.414595:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300145792 00000100:00100000:0.0:1782488345.414644:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455218c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300145536:12345-0@lo:400:kworker.0 Request processed in 75us (207us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488345.414652:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 611 00000100:00100000:0.0:1782488345.414700:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579576680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300145536:0@lo:400:kworker.0 00000100:00100000:2.0:1782488345.414758:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14060 00000100:00100000:2.0:1782488345.414765:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300145664:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488345.414780:0:190296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488345.414817:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300145664:12345-0@lo:38:kworker.0 Request processed in 51us (317us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488345.418901:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14061 00000100:00100000:0.0:1782488345.418916:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300145792:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488345.418939:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488345.419026:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300145792:12345-0@lo:38:kworker.0 Request processed in 113us (4480us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488345.419034:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14061 00000100:00100000:2.0:1782488345.428789:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14060 00000100:00080000:2.0:1782488345.428880:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488345.428887:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488345.428891:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579574700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300145664:0@lo:38:kworker.0 00000100:00080000:2.0:1782488345.428905:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488345.428908:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488345.428911:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579577b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300145792:0@lo:38:kworker.0 00010000:00010000:1.0:1782488346.244186:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488347.265801:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488348.294946:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488349.316770:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488350.338428:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488350.531131:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488350.531141:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488350.531145:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488350.531165:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7972 00000100:00080000:2.0:1782488350.531170:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488350.531178:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488350.531202:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff1180 x1869066300146048/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488350.531222:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488350.531239:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488350.531241:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488350.531246:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7972 00000100:00080000:2.0:1782488350.531249:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488350.531254:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488350.531269:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff1880 x1869066300146176/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488350.531332:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1782488350.531350:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300145920:0@lo:400:kworker.0 00000100:00100000:2.0:1782488350.531427:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488350.531510:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300146048:0@lo:38:kworker.0 00000100:00100000:2.0:1782488350.531545:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488350.531582:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300146176:0@lo:38:kworker.0 00000100:00100000:0.0:1782488350.533361:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300145920 00000100:00100000:0.0:1782488350.533386:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300145920 00000100:00100000:0.0:1782488350.533440:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 612 00000100:00100000:0.0:1782488350.533519:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455218450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300145920:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488350.534178:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455218450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300145920:12345-0@lo:400:kworker.0 Request processed in 658us (2749us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488350.534187:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 612 00000100:00100000:2.0:1782488350.535456:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488350.535538:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300146048 00000100:00100000:2.0:1782488350.535547:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300146048 00000100:00100000:2.0:1782488350.535596:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300146176 00000100:00100000:2.0:1782488350.535600:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300146176 00000100:00100000:2.0:1782488350.535640:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14062 00000100:00100000:2.0:1782488350.535647:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300146048:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488350.535665:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488350.535726:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300146048:12345-0@lo:38:kworker.0 Request processed in 76us (4179us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488350.535735:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14062 00000100:00100000:2.0:1782488350.535755:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14063 00000100:00100000:2.0:1782488350.535760:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300146176:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488350.535782:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488350.535816:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300146176:12345-0@lo:38:kworker.0 Request processed in 54us (4207us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488350.535823:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14063 00000100:00100000:2.0:1782488350.535872:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300145920:0@lo:400:kworker.0 00000100:00080000:2.0:1782488350.535900:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488350.535905:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488350.535908:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300146048:0@lo:38:kworker.0 00000100:00080000:2.0:1782488350.535917:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488350.535920:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488350.535923:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300146176:0@lo:38:kworker.0 00010000:00010000:0.0:1782488351.370852:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488352.385870:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488353.412889:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488354.434080:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488355.459387:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488355.649759:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488355.649768:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488355.649772:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488355.649779:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7977 00000100:00080000:2.0:1782488355.649784:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488355.649791:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488355.649828:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff1180 x1869066300146432/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488355.649845:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488355.649849:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488355.649850:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488355.649854:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7977 00000100:00080000:2.0:1782488355.649856:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488355.649875:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488355.649889:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff2300 x1869066300146560/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488355.649914:0:3315:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1782488355.649942:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300146304:0@lo:400:kworker.0 00000100:00100000:2.0:1782488355.650014:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488355.650076:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300146432:0@lo:38:kworker.0 00000100:00100000:2.0:1782488355.650105:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488355.650125:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300146560:0@lo:38:kworker.0 00000100:00100000:2.0:1782488355.650144:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488355.650176:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300146304 00000100:00100000:2.0:1782488355.650183:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300146304 00000100:00100000:2.0:1782488355.650230:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 613 00000100:00100000:2.0:1782488355.650239:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45521a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300146304:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488355.650290:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45521a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300146304:12345-0@lo:400:kworker.0 Request processed in 51us (275us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488355.650568:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 613 00000100:00100000:2.0:1782488355.650629:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300146432 00000100:00100000:2.0:1782488355.650633:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300146432 00000100:00100000:2.0:1782488355.650654:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300146560 00000100:00100000:2.0:1782488355.650657:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300146560 00000100:00100000:2.0:1782488355.650672:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14064 00000100:00100000:2.0:1782488355.650677:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300146432:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488355.650692:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488355.650728:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300146432:12345-0@lo:38:kworker.0 Request processed in 49us (622us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488355.650735:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14064 00000100:00100000:2.0:1782488355.650758:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14065 00000100:00100000:2.0:1782488355.650762:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300146560:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488355.650770:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488355.650858:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300146560:12345-0@lo:38:kworker.0 Request processed in 92us (712us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488355.650864:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14065 00000100:00080000:2.0:1782488355.650913:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488355.650929:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488355.650932:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300146432:0@lo:38:kworker.0 00000100:00080000:2.0:1782488355.650942:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488355.650945:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488355.650948:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300146560:0@lo:38:kworker.0 00000100:00100000:2.0:1782488355.650977:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300146304:0@lo:400:kworker.0 00010000:00010000:3.0:1782488356.485886:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488357.511657:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488358.533834:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488359.555575:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488360.578843:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488360.770109:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488360.770116:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488360.770120:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488360.770128:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7983 00000100:00080000:2.0:1782488360.770133:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488360.770139:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488360.770162:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff2300 x1869066300146816/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488360.770179:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488360.770182:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488360.770184:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1782488360.770190:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1782488360.770219:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300146688:0@lo:400:kworker.0 00000100:00080000:2.0:1782488360.770219:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7983 00000100:00080000:2.0:1782488360.770222:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488360.770226:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488360.770239:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff1180 x1869066300146944/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488360.770263:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300146816:0@lo:38:kworker.0 00000100:00100000:2.0:1782488360.775216:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488360.775318:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300146944:0@lo:38:kworker.0 00000100:00100000:2.0:1782488360.775362:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488360.777552:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488360.777769:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300146816 00000100:00100000:0.0:1782488360.777777:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300146816 00000100:00100000:0.0:1782488360.777822:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300146944 00000100:00100000:0.0:1782488360.777824:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300146944 00000100:00100000:0.0:1782488360.778181:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14066 00000100:00100000:0.0:1782488360.778190:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300146816:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488360.778217:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488360.780079:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300146688 00000100:00100000:2.0:1782488360.780101:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300146688 00000100:00100000:2.0:1782488360.780153:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 614 00000100:00100000:2.0:1782488360.780161:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579add850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300146688:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488360.780249:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579add850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300146688:12345-0@lo:400:kworker.0 Request processed in 88us (2703us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488360.780257:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 614 00000100:00100000:0.0:1782488360.795998:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300146816:12345-0@lo:38:kworker.0 Request processed in 17800us (20781us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488360.796013:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14066 00000100:00100000:0.0:1782488360.796047:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14067 00000100:00100000:0.0:1782488360.796053:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300146944:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488360.796069:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488360.796094:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300146944:12345-0@lo:38:kworker.0 Request processed in 40us (20731us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488360.796099:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14067 00000100:00080000:0.0:1782488360.796670:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488360.796733:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488360.796737:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300146816:0@lo:38:kworker.0 00000100:00080000:0.0:1782488360.796753:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488360.796756:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488360.796760:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300146944:0@lo:38:kworker.0 00000100:00100000:0.0:1782488360.796796:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff1880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300146688:0@lo:400:kworker.0 00010000:00010000:1.0:1782488361.605022:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488362.626056:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488363.650708:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488364.675852:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488365.702156:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488365.890601:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488365.890622:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488365.890626:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488365.890633:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7988 00000100:00080000:2.0:1782488365.890638:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488365.890644:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488365.890665:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff1c00 x1869066300147200/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488365.890703:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488365.890706:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488365.890708:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488365.890712:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7988 00000100:00080000:2.0:1782488365.890714:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488365.890718:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488365.890728:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff1f80 x1869066300147328/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488365.890801:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300147072:0@lo:400:kworker.0 00000100:00100000:2.0:1782488365.890887:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488365.890952:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300147200:0@lo:38:kworker.0 00000100:00100000:2.0:1782488365.890980:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488365.890999:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300147328:0@lo:38:kworker.0 00000100:00100000:2.0:1782488365.891019:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488365.891328:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300147072 00000100:00100000:2.0:1782488365.891335:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300147072 00000100:00100000:2.0:1782488365.891385:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 615 00000100:00100000:2.0:1782488365.891395:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579adc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300147072:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488365.891463:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579adc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300147072:12345-0@lo:400:kworker.0 Request processed in 69us (576us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488365.891470:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 615 00000100:00100000:2.0:1782488365.891514:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300147200 00000100:00100000:2.0:1782488365.891518:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300147200 00000100:00100000:2.0:1782488365.891536:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300147328 00000100:00100000:2.0:1782488365.891545:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300147328 00000100:00100000:2.0:1782488365.891562:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14068 00000100:00100000:2.0:1782488365.891567:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300147200:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488365.891581:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488365.891629:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300147200:12345-0@lo:38:kworker.0 Request processed in 60us (647us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488365.891635:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14068 00000100:00100000:2.0:1782488365.891646:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14069 00000100:00100000:2.0:1782488365.891650:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300147328:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488365.891656:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488365.891677:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300147328:12345-0@lo:38:kworker.0 Request processed in 26us (658us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488365.891682:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14069 00000100:00080000:2.0:1782488365.891727:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488365.891731:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488365.891734:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300147200:0@lo:38:kworker.0 00000100:00080000:2.0:1782488365.891742:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488365.891745:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488365.891747:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300147328:0@lo:38:kworker.0 00000100:00100000:2.0:1782488365.891773:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300147072:0@lo:400:kworker.0 00010000:00010000:3.0:1782488366.725753:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488367.746837:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488368.769870:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488369.796360:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488370.822451:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488371.081043:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488371.081051:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488371.081055:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488371.081063:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7993 00000100:00080000:2.0:1782488371.081083:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488371.081089:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488371.098391:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff1f80 x1869066300147584/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488371.098435:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488371.098440:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488371.098443:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488371.098450:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7993 00000100:00080000:2.0:1782488371.098454:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488371.098461:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488371.098499:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff1c00 x1869066300147712/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488371.098573:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300147456:0@lo:400:kworker.0 00000100:00100000:1.0:1782488371.100789:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300147584:0@lo:38:kworker.0 00000100:00100000:1.0:1782488371.100933:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488371.100992:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300147712:0@lo:38:kworker.0 00000100:00100000:1.0:1782488371.101016:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488371.101102:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300147584 00000100:00100000:1.0:1782488371.101110:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300147584 00000100:00100000:1.0:1782488371.101152:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300147712 00000100:00100000:1.0:1782488371.101156:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300147712 00000100:00100000:1.0:1782488371.101172:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14070 00000100:00100000:1.0:1782488371.101179:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb450138a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300147584:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488371.101196:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488371.101239:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb450138a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300147584:12345-0@lo:38:kworker.0 Request processed in 58us (305us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488371.101245:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14070 00000100:00100000:1.0:1782488371.101278:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14071 00000100:00100000:1.0:1782488371.101282:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb450138700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300147712:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488371.101289:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488371.101313:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb450138700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300147712:12345-0@lo:38:kworker.0 Request processed in 29us (296us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488371.101319:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14071 00000100:00080000:1.0:1782488371.101354:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488371.101360:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488371.101363:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff1f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300147584:0@lo:38:kworker.0 00000100:00080000:1.0:1782488371.101373:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488371.101376:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488371.101391:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300147712:0@lo:38:kworker.0 00000100:00100000:2.0:1782488371.151947:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488371.152051:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300147456 00000100:00100000:2.0:1782488371.152061:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300147456 00000100:00100000:2.0:1782488371.152109:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 616 00000100:00100000:2.0:1782488371.152732:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579adf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300147456:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488371.152793:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579adf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300147456:12345-0@lo:400:kworker.0 Request processed in 66us (854us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488371.152801:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 616 00000100:00100000:2.0:1782488371.153443:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300147456:0@lo:400:kworker.0 00010000:00010000:3.0:1782488371.846098:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488372.869468:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488373.897054:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488374.916954:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488375.937782:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1782488376.130838:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488376.130845:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488376.130849:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488376.130857:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 7998 00000100:00080000:1.0:1782488376.130873:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488376.130881:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488376.130904:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb450139880 x1869066300147968/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1782488376.130920:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488376.130923:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488376.130925:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488376.130929:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 7998 00000100:00080000:1.0:1782488376.130931:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488376.130947:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488376.130958:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb45013ad80 x1869066300148096/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1782488376.130990:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb450138380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300147840:0@lo:400:kworker.0 00000100:00100000:1.0:1782488376.131064:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488376.131167:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb450139880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300147968:0@lo:38:kworker.0 00000100:00100000:1.0:1782488376.131201:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488376.131222:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45013ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300148096:0@lo:38:kworker.0 00000100:00100000:1.0:1782488376.131247:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488376.131568:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300147840 00000100:00100000:2.0:1782488376.131577:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300147840 00000100:00100000:2.0:1782488376.131643:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 617 00000100:00100000:2.0:1782488376.131653:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579adf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300147840:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488376.131716:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579adf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300147840:12345-0@lo:400:kworker.0 Request processed in 64us (653us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488376.131723:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 617 00000100:00100000:2.0:1782488376.131782:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300147968 00000100:00100000:2.0:1782488376.131786:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300147968 00000100:00100000:2.0:1782488376.131805:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300148096 00000100:00100000:2.0:1782488376.131808:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300148096 00000100:00100000:2.0:1782488376.131821:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14072 00000100:00100000:2.0:1782488376.131826:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45013b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300147968:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488376.131842:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488376.131873:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45013b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300147968:12345-0@lo:38:kworker.0 Request processed in 45us (672us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488376.131879:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14072 00000100:00100000:2.0:1782488376.131890:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14073 00000100:00100000:2.0:1782488376.131893:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb450139500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300148096:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488376.131900:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488376.131922:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb450139500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300148096:12345-0@lo:38:kworker.0 Request processed in 27us (676us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488376.131927:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14073 00000100:00080000:2.0:1782488376.132379:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488376.132385:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488376.132389:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb450139880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300147968:0@lo:38:kworker.0 00000100:00080000:2.0:1782488376.132400:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488376.132594:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488376.132598:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45013ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300148096:0@lo:38:kworker.0 00000100:00100000:1.0:1782488376.132728:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb450138380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300147840:0@lo:400:kworker.0 00010000:00010000:1.0:1782488376.965111:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488377.991515:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488379.015864:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488380.038363:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488381.064128:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488381.250122:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488381.250129:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488381.250133:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488381.250140:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8003 00000100:00080000:0.0:1782488381.250144:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488381.250158:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488381.250181:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb579577b80 x1869066300148352/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1782488381.250198:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488381.250202:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488381.250204:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488381.250207:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8003 00000100:00080000:0.0:1782488381.250244:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488381.250249:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488381.250260:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb579574700 x1869066300148480/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488381.250282:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1782488381.250319:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579576680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300148224:0@lo:400:kworker.0 00000100:00100000:0.0:1782488381.250408:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488381.250471:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579577b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300148352:0@lo:38:kworker.0 00000100:00100000:0.0:1782488381.250498:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488381.250515:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579574700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300148480:0@lo:38:kworker.0 00000100:00100000:0.0:1782488381.250533:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488381.250574:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300148224 00000100:00100000:0.0:1782488381.250582:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300148224 00000100:00100000:0.0:1782488381.251232:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 618 00000100:00100000:0.0:1782488381.251246:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579adf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300148224:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488381.251456:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579adf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300148224:12345-0@lo:400:kworker.0 Request processed in 212us (1046us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488381.251465:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 618 00000100:00100000:0.0:1782488381.253360:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300148352 00000100:00100000:0.0:1782488381.253373:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300148352 00000100:00100000:0.0:1782488381.253442:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300148480 00000100:00100000:0.0:1782488381.253445:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300148480 00000100:00100000:0.0:1782488381.253472:0:213935:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14074 00000100:00100000:0.0:1782488381.253480:0:213935:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579575180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300148352:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488381.253504:0:213935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488381.253595:0:213935:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579575180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300148352:12345-0@lo:38:kworker.0 Request processed in 113us (3094us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488381.253645:0:213935:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14074 00000100:00100000:0.0:1782488381.253735:0:213935:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14075 00000100:00100000:0.0:1782488381.253740:0:213935:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579574000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300148480:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488381.253749:0:213935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488381.253954:0:213935:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579574000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300148480:12345-0@lo:38:kworker.0 Request processed in 211us (3419us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488381.253961:0:213935:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14075 00000100:00080000:0.0:1782488381.254023:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488381.254030:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488381.254034:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579577b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300148352:0@lo:38:kworker.0 00000100:00080000:0.0:1782488381.254044:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488381.254048:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488381.254051:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579574700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300148480:0@lo:38:kworker.0 00000100:00100000:0.0:1782488381.254082:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579576680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300148224:0@lo:400:kworker.0 00010000:00010000:3.0:1782488382.086301:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488383.105956:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488384.135650:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488385.154036:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488386.179652:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488386.371475:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488386.371484:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488386.371487:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488386.371496:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8008 00000100:00080000:2.0:1782488386.371500:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488386.371507:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488386.371528:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff1180 x1869066300148736/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488386.371545:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488386.371548:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488386.371550:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488386.371553:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8008 00000100:00080000:2.0:1782488386.371555:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488386.371558:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488386.371569:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff2300 x1869066300148864/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488386.371643:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300148608:0@lo:400:kworker.0 00000100:00100000:2.0:1782488386.371720:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488386.371804:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300148608 00000100:00100000:2.0:1782488386.371811:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300148608 00000100:00100000:2.0:1782488386.371851:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 619 00000100:00100000:2.0:1782488386.371860:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579addc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300148608:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488386.371909:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579addc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300148608:12345-0@lo:400:kworker.0 Request processed in 49us (188us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488386.371933:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 619 00000100:00100000:0.0:1782488386.372257:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300148736:0@lo:38:kworker.0 00000100:00100000:0.0:1782488386.372289:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488386.372307:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300148864:0@lo:38:kworker.0 00000100:00100000:0.0:1782488386.372321:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488386.372375:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300148608:0@lo:400:kworker.0 00000100:00100000:2.0:1782488386.381500:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300148736 00000100:00100000:2.0:1782488386.381514:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300148736 00000100:00100000:2.0:1782488386.381552:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300148864 00000100:00100000:2.0:1782488386.381555:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300148864 00000100:00100000:2.0:1782488386.381582:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14076 00000100:00100000:2.0:1782488386.381592:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579576680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300148736:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488386.381624:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488386.381713:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579576680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300148736:12345-0@lo:38:kworker.0 Request processed in 122us (9423us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488386.381721:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14076 00000100:00100000:2.0:1782488386.381738:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14077 00000100:00100000:2.0:1782488386.381742:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579574700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300148864:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488386.381749:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488386.381773:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579574700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300148864:12345-0@lo:38:kworker.0 Request processed in 29us (9450us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488386.381779:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14077 00000100:00080000:2.0:1782488386.381826:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488386.381831:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488386.381835:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300148736:0@lo:38:kworker.0 00000100:00080000:2.0:1782488386.381845:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488386.381849:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488386.381851:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300148864:0@lo:38:kworker.0 00010000:00010000:3.0:1782488387.205797:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488388.227880:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488389.250629:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488390.273788:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488391.297777:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1782488391.501636:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488391.501650:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488391.501654:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488391.501663:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8013 00000100:00080000:1.0:1782488391.501668:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488391.501675:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488391.501699:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb45013bb80 x1869066300149120/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1782488391.501719:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488391.501723:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488391.501725:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488391.501729:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8013 00000100:00080000:1.0:1782488391.501731:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488391.501735:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488391.501747:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb45013b800 x1869066300149248/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1782488391.501769:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1782488391.501787:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb450138380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300148992:0@lo:400:kworker.0 00000100:00100000:1.0:1782488391.501914:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488391.501980:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45013bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300149120:0@lo:38:kworker.0 00000100:00100000:1.0:1782488391.502012:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488391.502034:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45013b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300149248:0@lo:38:kworker.0 00000100:00100000:1.0:1782488391.502052:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488391.502082:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300148992 00000100:00100000:1.0:1782488391.502089:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300148992 00000100:00100000:1.0:1782488391.502132:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 620 00000100:00100000:1.0:1782488391.502149:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579adfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300148992:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488391.502232:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579adfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300148992:12345-0@lo:400:kworker.0 Request processed in 85us (318us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488391.502239:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 620 00000100:00100000:1.0:1782488391.502295:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300149120 00000100:00100000:1.0:1782488391.502298:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300149120 00000100:00100000:1.0:1782488391.502319:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300149248 00000100:00100000:1.0:1782488391.502322:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300149248 00000100:00100000:1.0:1782488391.502335:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14078 00000100:00100000:1.0:1782488391.502341:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45013ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300149120:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488391.502358:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488391.502389:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45013ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300149120:12345-0@lo:38:kworker.0 Request processed in 47us (377us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488391.502395:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14078 00000100:00100000:1.0:1782488391.502406:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14079 00000100:00100000:1.0:1782488391.502410:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb450139880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300149248:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488391.502417:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488391.502439:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb450139880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300149248:12345-0@lo:38:kworker.0 Request processed in 27us (385us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488391.502445:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14079 00000100:00080000:1.0:1782488391.502490:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488391.502495:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488391.502498:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45013bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300149120:0@lo:38:kworker.0 00000100:00080000:1.0:1782488391.502508:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488391.502511:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488391.502513:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45013b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300149248:0@lo:38:kworker.0 00000100:00100000:1.0:1782488391.502560:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb450138380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300148992:0@lo:400:kworker.0 00010000:00010000:1.0:1782488392.329125:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488393.349000:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488394.370131:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488395.396339:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488396.421438:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1782488396.609739:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488396.609747:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488396.609751:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488396.609760:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8018 00000100:00080000:1.0:1782488396.609764:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488396.609771:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488396.609792:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb45013b800 x1869066300149504/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1782488396.609808:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488396.609811:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488396.609813:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488396.609816:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8018 00000100:00080000:1.0:1782488396.609819:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488396.609822:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488396.609833:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb45013bb80 x1869066300149632/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1782488396.609858:0:3315:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1782488396.609875:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb450138380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300149376:0@lo:400:kworker.0 00000100:00100000:1.0:1782488396.610076:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488396.610141:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45013b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300149504:0@lo:38:kworker.0 00000100:00100000:1.0:1782488396.610187:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488396.610205:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45013bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300149632:0@lo:38:kworker.0 00000100:00100000:1.0:1782488396.610243:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488396.610287:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300149376 00000100:00100000:1.0:1782488396.610294:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300149376 00000100:00100000:1.0:1782488396.610338:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 621 00000100:00100000:1.0:1782488396.610348:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579adc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300149376:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488396.610396:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579adc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300149376:12345-0@lo:400:kworker.0 Request processed in 49us (319us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488396.610406:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 621 00000100:00100000:1.0:1782488396.610445:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300149504 00000100:00100000:1.0:1782488396.610449:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300149504 00000100:00100000:1.0:1782488396.610470:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300149632 00000100:00100000:1.0:1782488396.610474:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300149632 00000100:00100000:2.0:1782488396.610600:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14080 00000100:00100000:2.0:1782488396.610625:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45034aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300149504:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488396.610641:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488396.610680:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45034aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300149504:12345-0@lo:38:kworker.0 Request processed in 54us (491us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488396.610687:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14080 00000100:00080000:2.0:1782488396.610724:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488396.610730:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488396.610735:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45013b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300149504:0@lo:38:kworker.0 00000100:00100000:1.0:1782488396.610867:0:213935:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14081 00000100:00100000:1.0:1782488396.610873:0:213935:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb450349180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300149632:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488396.610882:0:213935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488396.610913:0:213935:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb450349180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300149632:12345-0@lo:38:kworker.0 Request processed in 38us (670us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488396.610920:0:213935:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14081 00000100:00100000:1.0:1782488396.610967:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb450138380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300149376:0@lo:400:kworker.0 00000100:00080000:2.0:1782488396.611105:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488396.611111:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488396.611115:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45013bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300149632:0@lo:38:kworker.0 00010000:00010000:0.0:1782488397.441776:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488398.466995:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488399.496652:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488400.520250:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488401.547170:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488401.730681:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488401.730689:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488401.730693:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488401.730702:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8024 00000100:00080000:2.0:1782488401.730708:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488401.730715:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488401.730740:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff1180 x1869066300149888/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488401.730759:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488401.730763:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488401.730766:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488401.730770:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8024 00000100:00080000:2.0:1782488401.730773:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488401.730777:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488401.730856:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff2d80 x1869066300150016/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488401.730886:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1782488401.730905:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300149760:0@lo:400:kworker.0 00000100:00100000:2.0:1782488401.730989:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488401.731058:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300149888:0@lo:38:kworker.0 00000100:00100000:1.0:1782488401.731059:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300149760 00000100:00100000:1.0:1782488401.731070:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300149760 00000100:00100000:2.0:1782488401.731097:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488401.731136:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300150016:0@lo:38:kworker.0 00000100:00100000:1.0:1782488401.731139:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 622 00000100:00100000:1.0:1782488401.731153:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb454603850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300149760:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488401.731168:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488401.731211:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300149888 00000100:00100000:2.0:1782488401.731215:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300149888 00000100:00100000:1.0:1782488401.731238:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb454603850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300149760:12345-0@lo:400:kworker.0 Request processed in 87us (249us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488401.731241:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300150016 00000100:00100000:2.0:1782488401.731246:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300150016 00000100:00100000:1.0:1782488401.731252:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 622 00000100:00100000:2.0:1782488401.731272:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14082 00000100:00100000:2.0:1782488401.731280:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300149888:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488401.731297:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488401.731350:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300149888:12345-0@lo:38:kworker.0 Request processed in 67us (253us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488401.731358:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14082 00000100:00100000:2.0:1782488401.731377:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14083 00000100:00100000:2.0:1782488401.731382:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300150016:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488401.731391:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488401.731422:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300150016:12345-0@lo:38:kworker.0 Request processed in 38us (255us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488401.731428:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14083 00000100:00080000:2.0:1782488401.731492:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488401.731498:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488401.731794:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300149888:0@lo:38:kworker.0 00000100:00080000:2.0:1782488401.731815:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488401.731820:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488401.731824:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300150016:0@lo:38:kworker.0 00000100:00100000:2.0:1782488401.732064:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300149760:0@lo:400:kworker.0 00010000:00010000:1.0:1782488402.571330:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488403.591185:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488404.617204:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488405.637193:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488406.666505:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488406.849895:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488406.849902:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488406.849906:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488406.849915:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8029 00000100:00080000:2.0:1782488406.849919:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488406.849926:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488406.849948:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff2d80 x1869066300150272/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488406.850083:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300150144:0@lo:400:kworker.0 00000100:00100000:0.0:1782488406.850196:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488406.850267:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300150144 00000100:00100000:0.0:1782488406.850274:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300150144 00000100:00080000:2.0:1782488406.850295:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488406.850299:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488406.850301:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488406.850305:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8029 00000100:00080000:2.0:1782488406.850307:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488406.850311:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488406.850318:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 623 00000100:00100000:2.0:1782488406.850325:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff1180 x1869066300150400/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488406.850326:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb454602c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300150144:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488406.850349:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300150272:0@lo:38:kworker.0 00000100:00100000:0.0:1782488406.850374:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb454602c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300150144:12345-0@lo:400:kworker.0 Request processed in 48us (177us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488406.850381:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 623 00000100:00100000:2.0:1782488406.850388:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488406.850421:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300150400:0@lo:38:kworker.0 00000100:00100000:0.0:1782488406.850428:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300150144:0@lo:400:kworker.0 00000100:00100000:2.0:1782488406.850441:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488406.850466:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300150272 00000100:00100000:2.0:1782488406.850469:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300150272 00000100:00100000:2.0:1782488406.850499:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300150400 00000100:00100000:2.0:1782488406.850502:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300150400 00000100:00100000:2.0:1782488406.850520:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14084 00000100:00100000:2.0:1782488406.850525:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45013bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300150272:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488406.851622:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488406.852163:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14085 00000100:00100000:0.0:1782488406.852265:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb450138380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300150400:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488406.852277:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488406.852336:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb450138380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300150400:12345-0@lo:38:kworker.0 Request processed in 164us (1893us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488406.852344:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14085 00000100:00100000:2.0:1782488406.852743:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45013bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300150272:12345-0@lo:38:kworker.0 Request processed in 2216us (2352us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488406.852751:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14084 00000100:00080000:2.0:1782488406.852794:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488406.852799:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488406.852802:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300150272:0@lo:38:kworker.0 00000100:00080000:2.0:1782488406.852814:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488406.852817:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488406.852820:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300150400:0@lo:38:kworker.0 00010000:00010000:1.0:1782488407.686682:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488408.708470:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488409.730627:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488410.757534:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488411.786325:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488411.969796:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488411.969804:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488411.969809:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488411.969817:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8034 00000100:00080000:2.0:1782488411.969823:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488411.969830:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488411.969860:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb547781880 x1869066300150656/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488411.969937:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488411.969941:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488411.969944:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488411.969949:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8034 00000100:00080000:2.0:1782488411.969952:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488411.969957:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488411.969987:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb452cc9f80 x1869066300150784/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488411.970017:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1782488411.970170:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45013b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300150528:0@lo:400:kworker.0 00000100:00100000:2.0:1782488411.970243:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488411.970306:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb547781880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300150656:0@lo:38:kworker.0 00000100:00100000:2.0:1782488411.970352:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488411.970376:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300150784:0@lo:38:kworker.0 00000100:00100000:2.0:1782488411.970404:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488411.970465:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300150528 00000100:00100000:2.0:1782488411.970472:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300150528 00000100:00100000:2.0:1782488411.970545:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 624 00000100:00100000:2.0:1782488411.970556:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb454601850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300150528:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488411.970627:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb454601850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300150528:12345-0@lo:400:kworker.0 Request processed in 55us (367us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488411.970636:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 624 00000100:00100000:2.0:1782488411.970675:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300150656 00000100:00100000:2.0:1782488411.970680:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300150656 00000100:00100000:2.0:1782488411.970728:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300150784 00000100:00100000:2.0:1782488411.970732:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300150784 00000100:00100000:2.0:1782488411.970755:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14086 00000100:00100000:2.0:1782488411.970762:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb452cc8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300150656:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488411.970790:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488411.970830:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb452cc8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300150656:12345-0@lo:38:kworker.0 Request processed in 67us (476us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488411.970838:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14086 00000100:00100000:2.0:1782488411.970854:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14087 00000100:00100000:2.0:1782488411.970860:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb452ccbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300150784:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488411.970868:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488411.970897:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb452ccbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300150784:12345-0@lo:38:kworker.0 Request processed in 36us (494us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488411.970905:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14087 00000100:00080000:2.0:1782488411.970964:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488411.970970:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488411.970975:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb547781880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300150656:0@lo:38:kworker.0 00000100:00080000:2.0:1782488411.970991:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488411.970995:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488411.970999:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300150784:0@lo:38:kworker.0 00000100:00100000:2.0:1782488411.971281:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45013b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300150528:0@lo:400:kworker.0 00010000:00010000:1.0:1782488412.802696:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488413.830073:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488414.851434:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488415.874897:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488416.902557:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488417.091327:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488417.091351:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488417.091355:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488417.091363:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8039 00000100:00080000:2.0:1782488417.091387:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488417.091393:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488417.091414:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb452ccad80 x1869066300151040/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488417.091431:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488417.091434:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488417.091436:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488417.091439:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8039 00000100:00080000:2.0:1782488417.091442:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488417.091445:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488417.091453:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb452ccaa00 x1869066300151168/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488417.091480:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300150912:0@lo:400:kworker.0 00000100:00100000:2.0:1782488417.091565:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488417.091663:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300151040:0@lo:38:kworker.0 00000100:00100000:1.0:1782488417.092545:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300150912 00000100:00100000:1.0:1782488417.092574:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300150912 00000100:00100000:1.0:1782488417.092675:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 625 00000100:00100000:1.0:1782488417.092685:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb454602450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300150912:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488417.092876:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb454602450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300150912:12345-0@lo:400:kworker.0 Request processed in 192us (1310us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488417.092884:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 625 00000100:00100000:2.0:1782488417.092896:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488417.092940:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452cc9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300150912:0@lo:400:kworker.0 00000100:00100000:2.0:1782488417.092955:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300151168:0@lo:38:kworker.0 00000100:00100000:1.0:1782488417.092998:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300151040 00000100:00100000:1.0:1782488417.093002:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300151040 00000100:00100000:2.0:1782488417.093006:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488417.099483:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300151168 00000100:00100000:1.0:1782488417.099498:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300151168 00000100:00100000:1.0:1782488417.099541:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14088 00000100:00100000:1.0:1782488417.099549:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb452cc8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300151040:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488417.099582:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488417.099681:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb452cc8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300151040:12345-0@lo:38:kworker.0 Request processed in 131us (6784us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488417.099688:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14088 00000100:00100000:1.0:1782488417.099710:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14089 00000100:00100000:1.0:1782488417.099714:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb452cc9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300151168:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488417.099752:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488417.100488:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb452cc9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300151168:12345-0@lo:38:kworker.0 Request processed in 769us (7477us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488417.100497:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14089 00000100:00080000:1.0:1782488417.100554:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488417.100559:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488417.100563:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452ccad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300151040:0@lo:38:kworker.0 00000100:00080000:1.0:1782488417.100572:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488417.100575:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488417.100577:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452ccaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300151168:0@lo:38:kworker.0 00010000:00010000:1.0:1782488417.923426:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488418.952914:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488419.972557:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488421.009020:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488422.017888:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1782488422.218122:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488422.218129:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488422.218133:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488422.218141:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8044 00000100:00080000:1.0:1782488422.218146:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488422.218153:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488422.218206:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb450348a80 x1869066300151424/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1782488422.218223:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488422.218227:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488422.218228:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488422.218232:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8044 00000100:00080000:1.0:1782488422.218271:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488422.218276:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488422.218289:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb45034a300 x1869066300151552/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1782488422.218317:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1782488422.218346:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb450348380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300151296:0@lo:400:kworker.0 00000100:00100000:1.0:1782488422.218437:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488422.218507:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb450348a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300151424:0@lo:38:kworker.0 00000100:00100000:1.0:1782488422.218554:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488422.218574:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45034a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300151552:0@lo:38:kworker.0 00000100:00100000:1.0:1782488422.218595:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488422.218675:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300151296 00000100:00100000:1.0:1782488422.218682:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300151296 00000100:00100000:1.0:1782488422.218724:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 626 00000100:00100000:1.0:1782488422.218732:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb454600850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300151296:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488422.218778:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb454600850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300151296:12345-0@lo:400:kworker.0 Request processed in 47us (340us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488422.218784:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 626 00000100:00100000:1.0:1782488422.218843:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300151424 00000100:00100000:1.0:1782488422.218846:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300151424 00000100:00100000:1.0:1782488422.218862:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300151552 00000100:00100000:1.0:1782488422.218864:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300151552 00000100:00100000:1.0:1782488422.218879:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14090 00000100:00100000:1.0:1782488422.218884:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb450349500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300151424:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488422.218898:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488422.218930:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb450349500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300151424:12345-0@lo:38:kworker.0 Request processed in 44us (374us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488422.218955:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14090 00000100:00100000:1.0:1782488422.218968:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14091 00000100:00100000:0.0:1782488422.219106:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb450348380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300151296:0@lo:400:kworker.0 00000100:00080000:0.0:1782488422.219175:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488422.219199:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488422.219202:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb450348a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300151424:0@lo:38:kworker.0 00000100:00100000:1.0:1782488422.221059:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb450349c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300151552:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488422.221086:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488422.221156:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb450349c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300151552:12345-0@lo:38:kworker.0 Request processed in 2181us (2559us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488422.221165:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14091 00000100:00080000:0.0:1782488422.221278:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488422.221284:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488422.221288:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45034a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300151552:0@lo:38:kworker.0 00010000:00010000:1.0:1782488423.041743:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488424.066776:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488425.095705:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488426.115744:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488427.138591:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1782488427.330445:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488427.330452:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488427.330455:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488427.330463:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8049 00000100:00080000:1.0:1782488427.330468:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488427.330475:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:3.0:1782488427.330479:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb450349f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300151680:0@lo:400:kworker.0 00000100:00100000:1.0:1782488427.330506:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb450348e00 x1869066300151808/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1782488427.330523:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488427.330527:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488427.330528:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488427.330531:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8049 00000100:00080000:1.0:1782488427.330533:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488427.330536:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:3.0:1782488427.330591:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488427.330682:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300151680 00000100:00100000:3.0:1782488427.330689:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300151680 00000100:00100000:1.0:1782488427.330728:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb45034bb80 x1869066300151936/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1782488427.330731:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 627 00000100:00100000:3.0:1782488427.330739:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb454603c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300151680:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488427.330750:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb450348e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300151808:0@lo:38:kworker.0 00000100:00100000:3.0:1782488427.331285:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb454603c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300151680:12345-0@lo:400:kworker.0 Request processed in 545us (694us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488427.331294:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 627 00000100:00100000:3.0:1782488427.332837:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb450349f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300151680:0@lo:400:kworker.0 00000100:00100000:1.0:1782488427.338364:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488427.338429:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45034bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300151936:0@lo:38:kworker.0 00000100:00100000:1.0:1782488427.338483:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488427.338622:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300151808 00000100:00100000:1.0:1782488427.338630:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300151808 00000100:00100000:1.0:1782488427.338687:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300151936 00000100:00100000:1.0:1782488427.338690:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300151936 00000100:00100000:1.0:1782488427.338708:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14092 00000100:00100000:1.0:1782488427.338715:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45034b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300151808:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488427.338732:0:190296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488427.338770:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45034b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300151808:12345-0@lo:38:kworker.0 Request processed in 54us (414us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488427.338778:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14092 00000100:00100000:1.0:1782488427.338795:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14093 00000100:00100000:1.0:1782488427.338799:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb450348000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300151936:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488427.338806:0:190296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488427.338825:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb450348000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300151936:12345-0@lo:38:kworker.0 Request processed in 24us (341us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488427.338830:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14093 00000100:00080000:1.0:1782488427.338866:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488427.338871:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488427.338875:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb450348e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300151808:0@lo:38:kworker.0 00000100:00080000:1.0:1782488427.338883:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488427.338925:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488427.338929:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45034bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300151936:0@lo:38:kworker.0 00010000:00010000:3.0:1782488428.178621:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488429.200904:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488430.225436:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488431.234796:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488432.259582:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488432.450554:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488432.450585:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488432.450589:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488432.450597:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8054 00000100:00080000:2.0:1782488432.450601:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488432.450620:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488432.450654:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb452cc8380 x1869066300152192/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488432.450673:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488432.450676:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488432.450678:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488432.450681:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8054 00000100:00080000:2.0:1782488432.450684:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488432.450687:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488432.450697:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb452cc8a80 x1869066300152320/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488432.450736:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300152064:0@lo:400:kworker.0 00000100:00100000:2.0:1782488432.450822:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488432.450890:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300152192:0@lo:38:kworker.0 00000100:00100000:2.0:1782488432.450917:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488432.450934:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300152320:0@lo:38:kworker.0 00000100:00100000:2.0:1782488432.450950:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488432.450976:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300152064 00000100:00100000:2.0:1782488432.450982:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300152064 00000100:00100000:2.0:1782488432.451021:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 628 00000100:00100000:2.0:1782488432.451028:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb454601450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300152064:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488432.451068:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb454601450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300152064:12345-0@lo:400:kworker.0 Request processed in 40us (246us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488432.451074:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 628 00000100:00100000:2.0:1782488432.451115:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300152192 00000100:00100000:2.0:1782488432.451118:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300152192 00000100:00100000:2.0:1782488432.451211:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300152320 00000100:00100000:2.0:1782488432.451215:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300152320 00000100:00100000:2.0:1782488432.451231:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14094 00000100:00100000:2.0:1782488432.451235:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb452ccb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300152192:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488432.451250:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488432.451283:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb452ccb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300152192:12345-0@lo:38:kworker.0 Request processed in 46us (364us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488432.451289:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14094 00000100:00100000:2.0:1782488432.451301:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14095 00000100:00100000:2.0:1782488432.451305:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb452cca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300152320:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488432.451312:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488432.451334:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb452cca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300152320:12345-0@lo:38:kworker.0 Request processed in 27us (382us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488432.451339:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14095 00000100:00080000:2.0:1782488432.451382:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488432.451387:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488432.451390:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300152192:0@lo:38:kworker.0 00000100:00080000:2.0:1782488432.451398:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488432.451400:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488432.451403:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300152320:0@lo:38:kworker.0 00000100:00100000:2.0:1782488432.452512:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300152064:0@lo:400:kworker.0 00010000:00010000:1.0:1782488433.284797:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488434.305948:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488435.331643:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488436.363682:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488437.382052:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488437.570373:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488437.570380:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488437.570385:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488437.570393:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8059 00000100:00080000:0.0:1782488437.570397:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488437.570404:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488437.570427:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb579575c00 x1869066300152576/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1782488437.570444:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488437.570448:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488437.570450:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488437.570453:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8059 00000100:00080000:0.0:1782488437.570455:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488437.570459:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488437.570470:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb579576a00 x1869066300152704/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488437.570491:0:3315:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1782488437.570507:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579577b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300152448:0@lo:400:kworker.0 00000100:00100000:0.0:1782488437.570713:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488437.570770:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579575c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300152576:0@lo:38:kworker.0 00000100:00100000:0.0:1782488437.570811:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488437.570833:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579576a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300152704:0@lo:38:kworker.0 00000100:00100000:0.0:1782488437.570855:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488437.570905:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300152448 00000100:00100000:0.0:1782488437.570912:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300152448 00000100:00100000:0.0:1782488437.570958:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 629 00000100:00100000:0.0:1782488437.570967:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb454602850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300152448:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488437.571016:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb454602850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300152448:12345-0@lo:400:kworker.0 Request processed in 50us (303us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488437.571024:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 629 00000100:00100000:0.0:1782488437.571060:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300152576 00000100:00100000:0.0:1782488437.571064:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300152576 00000100:00100000:0.0:1782488437.571083:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300152704 00000100:00100000:0.0:1782488437.571086:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300152704 00000100:00100000:0.0:1782488437.571104:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14096 00000100:00100000:0.0:1782488437.571108:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579577480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300152576:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488437.571122:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488437.571155:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579577480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300152576:12345-0@lo:38:kworker.0 Request processed in 45us (354us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488437.571162:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14096 00000100:00100000:0.0:1782488437.571174:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14097 00000100:00100000:0.0:1782488437.571178:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579575f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300152704:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488437.571184:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488437.571207:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579575f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300152704:12345-0@lo:38:kworker.0 Request processed in 28us (352us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488437.571238:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14097 00000100:00080000:0.0:1782488437.571287:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488437.571292:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488437.571296:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579575c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300152576:0@lo:38:kworker.0 00000100:00080000:0.0:1782488437.571305:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488437.571308:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488437.571311:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579576a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300152704:0@lo:38:kworker.0 00000100:00100000:0.0:1782488437.571340:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579577b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300152448:0@lo:400:kworker.0 00010000:00010000:1.0:1782488438.403913:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488439.439016:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488440.452876:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488441.476273:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488442.506943:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1782488442.690912:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488442.690919:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488442.690922:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488442.690930:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8064 00000100:00080000:1.0:1782488442.690934:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488442.690940:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488442.690961:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb450348e00 x1869066300152960/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1782488442.690978:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488442.690981:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488442.690983:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488442.690986:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8064 00000100:00080000:1.0:1782488442.690988:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488442.690991:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488442.690994:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45034bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300152832:0@lo:400:kworker.0 00000100:00100000:1.0:1782488442.691002:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb45034ad80 x1869066300153088/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1782488442.691045:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb450348e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300152960:0@lo:38:kworker.0 00000100:00100000:0.0:1782488442.691065:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488442.691074:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488442.691154:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300152832 00000100:00100000:0.0:1782488442.691162:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300152832 00000100:00100000:0.0:1782488442.691205:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 630 00000100:00100000:0.0:1782488442.691213:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb454600050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300152832:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488442.691275:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb454600050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300152832:12345-0@lo:400:kworker.0 Request processed in 62us (209us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488442.691282:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 630 00000100:00100000:0.0:1782488442.691329:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45034bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300152832:0@lo:400:kworker.0 00000100:00100000:0.0:1782488442.691434:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300152960 00000100:00100000:0.0:1782488442.691439:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300152960 00000100:00100000:0.0:1782488442.691462:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14098 00000100:00100000:0.0:1782488442.691467:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45034a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300152960:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488442.691481:0:190296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488442.691562:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45034a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300152960:12345-0@lo:38:kworker.0 Request processed in 93us (485us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488442.691570:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14098 00000100:00080000:1.0:1782488442.707415:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488442.707429:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488442.707434:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb450348e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300152960:0@lo:38:kworker.0 00000100:00100000:1.0:1782488442.707447:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45034ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300153088:0@lo:38:kworker.0 00000100:00100000:1.0:1782488442.707510:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488442.707719:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300153088 00000100:00100000:0.0:1782488442.707728:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300153088 00000100:00100000:0.0:1782488442.707782:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14099 00000100:00100000:0.0:1782488442.707790:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb450348700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300153088:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488442.707808:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488442.707852:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb450348700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300153088:12345-0@lo:38:kworker.0 Request processed in 62us (342us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488442.707868:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14099 00000100:00080000:1.0:1782488442.708095:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488442.708099:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488442.708101:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45034ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300153088:0@lo:38:kworker.0 00010000:00010000:0.0:1782488443.534920:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488444.546350:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488445.576362:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488446.604138:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488447.625648:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488447.818549:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488447.818556:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488447.818560:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488447.818579:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8070 00000100:00080000:2.0:1782488447.818583:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488447.818589:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488447.818625:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb452cc8a80 x1869066300153344/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488447.818644:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488447.818647:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488447.818649:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488447.818652:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8070 00000100:00080000:2.0:1782488447.818655:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488447.818658:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488447.818668:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb452cc8380 x1869066300153472/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488447.818694:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300153216:0@lo:400:kworker.0 00000100:00100000:2.0:1782488447.818773:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488447.818848:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300153344:0@lo:38:kworker.0 00000100:00100000:2.0:1782488447.818879:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488447.818896:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300153472:0@lo:38:kworker.0 00000100:00100000:2.0:1782488447.818916:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488447.819677:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300153216 00000100:00100000:0.0:1782488447.819686:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300153216 00000100:00100000:0.0:1782488447.819740:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 631 00000100:00100000:0.0:1782488447.819750:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb454600c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300153216:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488447.819804:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb454600c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300153216:12345-0@lo:400:kworker.0 Request processed in 56us (1030us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488447.819819:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 631 00000100:00100000:0.0:1782488447.819850:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300153344 00000100:00100000:0.0:1782488447.819854:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300153344 00000100:00100000:0.0:1782488447.819869:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300153472 00000100:00100000:0.0:1782488447.819872:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300153472 00000100:00100000:0.0:1782488447.819886:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14100 00000100:00100000:0.0:1782488447.819891:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb452ccaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300153344:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488447.819905:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488447.819937:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb452ccaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300153344:12345-0@lo:38:kworker.0 Request processed in 44us (1057us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488447.819956:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14100 00000100:00100000:0.0:1782488447.819969:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14101 00000100:00100000:0.0:1782488447.819973:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb452ccad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300153472:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488447.819981:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488447.820030:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb452ccad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300153472:12345-0@lo:38:kworker.0 Request processed in 55us (1113us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488447.820035:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14101 00000100:00080000:2.0:1782488447.821011:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488447.821016:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488447.821019:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300153344:0@lo:38:kworker.0 00000100:00080000:2.0:1782488447.821029:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488447.821032:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488447.821060:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300153472:0@lo:38:kworker.0 00000100:00100000:2.0:1782488447.821094:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300153216:0@lo:400:kworker.0 00010000:00010000:1.0:1782488448.643675:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488449.667824:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488450.689731:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488451.719947:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488452.740200:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1782488452.929796:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300153600:0@lo:400:kworker.0 00000100:00100000:0.0:1782488452.929876:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1782488452.929946:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488452.929952:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488452.929955:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488452.929960:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8075 00000100:00080000:2.0:1782488452.929965:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488452.929972:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488452.929992:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb452cc8380 x1869066300153728/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488452.930007:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488452.930010:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488452.930011:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488452.930015:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8075 00000100:00080000:2.0:1782488452.930017:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488452.930020:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488452.930029:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb452cc8a80 x1869066300153856/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488452.930055:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300153728:0@lo:38:kworker.0 00000100:00100000:2.0:1782488452.930085:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488452.930104:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300153856:0@lo:38:kworker.0 00000100:00100000:2.0:1782488452.930122:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488452.930167:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300153728 00000100:00100000:2.0:1782488452.930189:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300153728 00000100:00100000:2.0:1782488452.930220:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300153856 00000100:00100000:2.0:1782488452.930223:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300153856 00000100:00100000:2.0:1782488452.930240:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14102 00000100:00100000:2.0:1782488452.930246:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb452cc9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300153728:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488452.930264:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488452.930300:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb452cc9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300153728:12345-0@lo:38:kworker.0 Request processed in 52us (214us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488452.930307:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14102 00000100:00100000:2.0:1782488452.930320:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14103 00000100:00100000:2.0:1782488452.930324:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45013b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300153856:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488452.930330:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488452.930347:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45013b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300153856:12345-0@lo:38:kworker.0 Request processed in 22us (225us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488452.930353:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14103 00000100:00080000:2.0:1782488452.930389:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488452.930393:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488452.930396:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300153728:0@lo:38:kworker.0 00000100:00080000:2.0:1782488452.930404:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488452.930406:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488452.930409:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300153856:0@lo:38:kworker.0 00000100:00100000:0.0:1782488452.936737:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300153600 00000100:00100000:0.0:1782488452.936750:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300153600 00000100:00100000:0.0:1782488452.936811:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 632 00000100:00100000:0.0:1782488452.936820:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb454601050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300153600:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488452.936904:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb454601050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300153600:12345-0@lo:400:kworker.0 Request processed in 84us (7028us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488452.936911:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 632 00000100:00100000:0.0:1782488452.936954:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300153600:0@lo:400:kworker.0 00010000:00010000:0.0:1782488453.762788:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488454.786880:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488455.810794:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488456.834042:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488457.859865:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1782488458.054238:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488458.054245:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488458.054249:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488458.054257:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8080 00000100:00080000:1.0:1782488458.054261:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488458.054270:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488458.054292:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb450348e00 x1869066300154112/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1782488458.054308:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488458.054324:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488458.054326:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488458.054329:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8080 00000100:00080000:1.0:1782488458.054331:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488458.054335:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488458.054346:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb45034b480 x1869066300154240/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1782488458.054373:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45034ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300153984:0@lo:400:kworker.0 00000100:00100000:1.0:1782488458.054447:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488458.054503:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb450348e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300154112:0@lo:38:kworker.0 00000100:00100000:1.0:1782488458.054528:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488458.054544:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45034b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300154240:0@lo:38:kworker.0 00000100:00100000:1.0:1782488458.054560:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488458.054635:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300153984 00000100:00100000:1.0:1782488458.054643:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300153984 00000100:00100000:1.0:1782488458.054686:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 633 00000100:00100000:1.0:1782488458.054694:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb454600450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300153984:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488458.054738:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb454600450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300153984:12345-0@lo:400:kworker.0 Request processed in 44us (291us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488458.054745:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 633 00000100:00100000:1.0:1782488458.054794:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45034ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300153984:0@lo:400:kworker.0 00000100:00100000:1.0:1782488458.054823:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300154112 00000100:00100000:1.0:1782488458.054826:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300154112 00000100:00100000:1.0:1782488458.054842:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300154240 00000100:00100000:1.0:1782488458.054844:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300154240 00000100:00100000:1.0:1782488458.054865:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14104 00000100:00100000:1.0:1782488458.054869:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45034bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300154112:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488458.054883:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488458.054911:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45034bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300154112:12345-0@lo:38:kworker.0 Request processed in 40us (381us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488458.054916:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14104 00000100:00100000:1.0:1782488458.054925:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14105 00000100:00100000:1.0:1782488458.054929:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb450349f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300154240:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488458.054935:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488458.054951:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb450349f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300154240:12345-0@lo:38:kworker.0 Request processed in 21us (390us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488458.054956:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14105 00000100:00080000:1.0:1782488458.054983:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488458.054987:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488458.054990:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb450348e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300154112:0@lo:38:kworker.0 00000100:00080000:1.0:1782488458.054996:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488458.054999:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488458.055001:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45034b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300154240:0@lo:38:kworker.0 00010000:00010000:1.0:1782488458.883677:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488459.905911:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488460.930277:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488461.955261:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488462.978238:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488463.170088:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488463.170095:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488463.170099:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488463.170108:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8085 00000100:00080000:2.0:1782488463.170157:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488463.170164:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488463.170186:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb452cc8a80 x1869066300154496/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488463.170202:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488463.170205:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488463.170207:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488463.170209:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8085 00000100:00080000:2.0:1782488463.170211:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488463.170215:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488463.170224:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb452cc8380 x1869066300154624/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1782488463.170454:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300154368:0@lo:400:kworker.0 00000100:00100000:1.0:1782488463.170542:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488463.171727:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300154496:0@lo:38:kworker.0 00000100:00100000:1.0:1782488463.171766:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488463.171787:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300154624:0@lo:38:kworker.0 00000100:00100000:1.0:1782488463.171821:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488463.171881:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300154496 00000100:00100000:1.0:1782488463.171889:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300154496 00000100:00100000:1.0:1782488463.171918:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300154624 00000100:00100000:1.0:1782488463.171921:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300154624 00000100:00100000:1.0:1782488463.171943:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14106 00000100:00100000:1.0:1782488463.171950:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45034b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300154496:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488463.171968:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488463.172009:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45034b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300154496:12345-0@lo:38:kworker.0 Request processed in 58us (243us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488463.172015:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14106 00000100:00100000:1.0:1782488463.172032:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14107 00000100:00100000:1.0:1782488463.172036:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb450348e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300154624:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488463.172041:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488463.172078:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb450348e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300154624:12345-0@lo:38:kworker.0 Request processed in 41us (256us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488463.172084:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14107 00000100:00080000:1.0:1782488463.172132:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488463.172137:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488463.172141:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300154496:0@lo:38:kworker.0 00000100:00080000:1.0:1782488463.172150:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488463.172154:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488463.172156:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300154624:0@lo:38:kworker.0 00000020:02000400:1.0:1782488463.175810:0:229334:0:(genops.c:1873:obd_exports_barrier()) lustre-MDT0000 is waiting for obd_unlinked_exports more than 128 seconds. The obd refcount = 5. Is it stuck? 00000100:00100000:2.0:1782488463.179640:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300154368 00000100:00100000:2.0:1782488463.179655:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300154368 00000100:00100000:2.0:1782488463.179720:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 634 00000100:00100000:2.0:1782488463.179729:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb454602050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300154368:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488463.179815:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb454602050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300154368:12345-0@lo:400:kworker.0 Request processed in 87us (9273us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488463.179823:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 634 00000100:00100000:2.0:1782488463.179880:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300154368:0@lo:400:kworker.0 00000020:02000400:1.0:1782488463.183746:0:229334:0:(genops.c:1840:print_export_data()) lustre-MDT0000: UNLINKED ffff9cb551430000 5c4ca15f-f34e-4111-b105-de67ffe3ec92 192.168.202.21@tcp 2 (0 0 0) 1 0 0 0: 0000000000000000 8589967277 stale:0 00010000:00010000:3.0:1782488464.004588:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488465.025736:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488466.051673:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488467.075485:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488468.100580:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488468.289770:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488468.289777:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488468.289782:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488468.289791:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8090 00000100:00080000:2.0:1782488468.289813:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488468.289821:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488468.289852:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb452cc8380 x1869066300154880/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488468.289871:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488468.289875:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488468.289877:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488468.289880:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8090 00000100:00080000:2.0:1782488468.289882:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488468.289885:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488468.289896:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb452cc8a80 x1869066300155008/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488468.289939:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1782488468.289962:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300154752:0@lo:400:kworker.0 00000100:00100000:0.0:1782488468.290127:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300154880:0@lo:38:kworker.0 00000100:00100000:0.0:1782488468.292261:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488468.292349:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300155008:0@lo:38:kworker.0 00000100:00100000:0.0:1782488468.292381:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488468.292468:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300154880 00000100:00100000:0.0:1782488468.292491:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300154880 00000100:00100000:0.0:1782488468.292547:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300155008 00000100:00100000:0.0:1782488468.292551:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300155008 00000100:00100000:0.0:1782488468.292572:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14108 00000100:00100000:0.0:1782488468.292580:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579577b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300154880:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488468.292598:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488468.292659:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579577b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300154880:12345-0@lo:38:kworker.0 Request processed in 79us (406us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488468.292667:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14108 00000100:00100000:0.0:1782488468.292684:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14109 00000100:00100000:0.0:1782488468.292688:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579576a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300155008:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488468.292695:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488468.292719:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579576a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300155008:12345-0@lo:38:kworker.0 Request processed in 29us (336us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488468.292724:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14109 00000100:00080000:0.0:1782488468.292770:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488468.292776:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488468.292780:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300154880:0@lo:38:kworker.0 00000100:00080000:0.0:1782488468.292790:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488468.292793:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488468.292796:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300155008:0@lo:38:kworker.0 00000100:00100000:2.0:1782488468.302316:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488468.309710:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300154752 00000100:00100000:0.0:1782488468.309729:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300154752 00000100:00100000:0.0:1782488468.309816:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 635 00000100:00100000:0.0:1782488468.309827:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb454603450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300154752:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488468.309926:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb454603450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300154752:12345-0@lo:400:kworker.0 Request processed in 101us (7617us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488468.309951:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 635 00000100:00100000:2.0:1782488468.310137:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300154752:0@lo:400:kworker.0 00010000:00010000:2.0:1782488469.121816:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488470.150683:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488471.171042:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488472.199735:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488473.218710:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1782488473.426044:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00080000:2.0:1782488473.426066:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488473.426072:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488473.426075:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488473.426083:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8095 00000100:00080000:2.0:1782488473.426088:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488473.426095:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488473.426119:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb452cc8a80 x1869066300155264/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488473.426149:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488473.426153:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488473.426155:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488473.426159:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8095 00000100:00080000:2.0:1782488473.426162:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488473.426166:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488473.426179:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb452cc8380 x1869066300155392/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488473.427085:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300155264:0@lo:38:kworker.0 00000100:00100000:0.0:1782488473.427153:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488473.427202:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300155392:0@lo:38:kworker.0 00000100:00100000:0.0:1782488473.427221:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488473.429122:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300155264 00000100:00100000:0.0:1782488473.429132:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300155264 00000100:00100000:0.0:1782488473.429171:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300155392 00000100:00100000:0.0:1782488473.429174:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300155392 00000100:00100000:0.0:1782488473.429196:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14110 00000100:00100000:0.0:1782488473.429203:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579575c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300155264:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488473.429223:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488473.429285:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579575c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300155264:12345-0@lo:38:kworker.0 Request processed in 80us (2131us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488473.429293:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14110 00000100:00100000:0.0:1782488473.429312:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14111 00000100:00100000:0.0:1782488473.429316:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579575500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300155392:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488473.429324:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488473.429348:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579575500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300155392:12345-0@lo:38:kworker.0 Request processed in 30us (2126us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488473.429354:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14111 00000100:00080000:0.0:1782488473.429412:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488473.429418:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488473.429422:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300155264:0@lo:38:kworker.0 00000100:00080000:0.0:1782488473.429432:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488473.429436:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488473.429455:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300155392:0@lo:38:kworker.0 00000100:00100000:1.0:1782488473.434111:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300155136:0@lo:400:kworker.0 00000100:00100000:1.0:1782488473.434192:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488473.445090:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300155136 00000100:00100000:3.0:1782488473.445139:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300155136 00000100:00100000:3.0:1782488473.446665:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 636 00000100:00100000:3.0:1782488473.446683:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb454601c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300155136:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782488473.446806:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb454601c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300155136:12345-0@lo:400:kworker.0 Request processed in 131us (12614us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488473.446814:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 636 00000100:00100000:1.0:1782488473.446918:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300155136:0@lo:400:kworker.0 00010000:00010000:1.0:1782488474.250156:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488475.272249:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488476.296457:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488477.315568:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488478.339685:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488478.529889:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488478.529897:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488478.529902:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488478.529910:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8100 00000100:00080000:2.0:1782488478.529915:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488478.529922:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488478.529943:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb452cc8380 x1869066300155648/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488478.529963:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488478.529967:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488478.529968:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488478.529972:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8100 00000100:00080000:2.0:1782488478.529974:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488478.529978:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488478.529990:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb452cc8a80 x1869066300155776/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488478.530048:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300155520:0@lo:400:kworker.0 00000100:00100000:2.0:1782488478.530119:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488478.530218:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300155648:0@lo:38:kworker.0 00000100:00100000:2.0:1782488478.530262:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488478.530320:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300155776:0@lo:38:kworker.0 00000100:00100000:2.0:1782488478.531120:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488478.531171:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300155520 00000100:00100000:2.0:1782488478.531179:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300155520 00000100:00100000:2.0:1782488478.531244:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 637 00000100:00100000:2.0:1782488478.531252:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb545d1b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300155520:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488478.531305:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb545d1b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300155520:12345-0@lo:400:kworker.0 Request processed in 53us (1185us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488478.531312:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 637 00000100:00100000:2.0:1782488478.531452:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300155648 00000100:00100000:2.0:1782488478.531458:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300155648 00000100:00100000:2.0:1782488478.531501:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300155776 00000100:00100000:2.0:1782488478.531505:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300155776 00000100:00100000:2.0:1782488478.531523:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14112 00000100:00100000:2.0:1782488478.531595:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb547781880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300155648:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488478.531641:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488478.531681:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb547781880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300155648:12345-0@lo:38:kworker.0 Request processed in 151us (1418us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488478.531688:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14112 00000100:00100000:2.0:1782488478.531701:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14113 00000100:00100000:2.0:1782488478.531705:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4bb705500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300155776:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488478.531727:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488478.531769:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4bb705500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300155776:12345-0@lo:38:kworker.0 Request processed in 62us (651us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488478.531776:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14113 00000100:00100000:2.0:1782488478.531828:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452ccb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300155520:0@lo:400:kworker.0 00000100:00080000:2.0:1782488478.531859:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488478.531864:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488478.531867:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452cc8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300155648:0@lo:38:kworker.0 00000100:00080000:2.0:1782488478.531875:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488478.531879:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488478.531882:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300155776:0@lo:38:kworker.0 00010000:00010000:1.0:1782488479.361759:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488480.388751:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488481.409808:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488482.434334:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488483.458014:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488483.649834:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488483.649841:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488483.649846:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488483.649855:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8105 00000100:00080000:2.0:1782488483.649860:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488483.649867:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488483.649892:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb4bb704700 x1869066300156032/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488483.649909:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488483.649912:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488483.649914:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488483.649917:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8105 00000100:00080000:2.0:1782488483.649961:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488483.649966:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488483.649980:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb4bb706300 x1869066300156160/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488483.650049:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4bb706680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300155904:0@lo:400:kworker.0 00000100:00100000:2.0:1782488483.650150:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488483.650247:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4bb704700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300156032:0@lo:38:kworker.0 00000100:00100000:2.0:1782488483.650296:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488483.650318:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4bb706300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300156160:0@lo:38:kworker.0 00000100:00100000:2.0:1782488483.650341:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488483.650384:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300155904 00000100:00100000:2.0:1782488483.650392:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300155904 00000100:00100000:2.0:1782488483.650460:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 638 00000100:00100000:2.0:1782488483.650470:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb545d19450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300155904:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488483.651491:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb545d19450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300155904:12345-0@lo:400:kworker.0 Request processed in 1017us (1336us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488483.651502:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 638 00000100:00100000:2.0:1782488483.651655:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300156032 00000100:00100000:2.0:1782488483.651661:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300156032 00000100:00100000:2.0:1782488483.651705:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300156160 00000100:00100000:2.0:1782488483.651709:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300156160 00000100:00100000:2.0:1782488483.651728:0:213935:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14114 00000100:00100000:2.0:1782488483.651734:0:213935:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4bb707b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300156032:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488483.651752:0:213935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488483.651794:0:213935:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4bb707b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300156032:12345-0@lo:38:kworker.0 Request processed in 59us (1498us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488483.651817:0:213935:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14114 00000100:00100000:2.0:1782488483.651830:0:213935:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14115 00000100:00100000:2.0:1782488483.651834:0:213935:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4bb705180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300156160:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488483.651841:0:213935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488483.651866:0:213935:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4bb705180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300156160:12345-0@lo:38:kworker.0 Request processed in 30us (1524us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488483.651872:0:213935:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14115 00000100:00080000:2.0:1782488483.651937:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488483.651942:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488483.651946:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4bb704700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300156032:0@lo:38:kworker.0 00000100:00080000:2.0:1782488483.651956:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488483.651959:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488483.651962:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4bb706300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300156160:0@lo:38:kworker.0 00000100:00100000:2.0:1782488483.651994:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4bb706680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300155904:0@lo:400:kworker.0 00010000:00010000:2.0:1782488484.482773:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488485.528217:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488486.537711:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488487.553776:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488488.581420:0:225729:0:(ldlm_pool.c:1332:ldlm_pools_recalc_task()) Negative interval(0) 00010000:00010000:3.0:1782488488.584688:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1782488488.770966:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488488.770972:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488488.770976:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488488.770984:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8111 00000100:00080000:1.0:1782488488.770989:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488488.770995:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488488.771016:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb45034a300 x1869066300156416/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1782488488.771032:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488488.771035:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488488.771037:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488488.771072:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8111 00000100:00080000:1.0:1782488488.771075:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488488.771079:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488488.771092:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb450348a80 x1869066300156544/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1782488488.771129:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1782488488.771145:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45034ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300156288:0@lo:400:kworker.0 00000100:00100000:1.0:1782488488.771246:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488488.771316:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45034a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300156416:0@lo:38:kworker.0 00000100:00100000:1.0:1782488488.771344:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488488.772738:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb450348a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300156544:0@lo:38:kworker.0 00000100:00100000:1.0:1782488488.772795:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488488.772996:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300156288 00000100:00100000:2.0:1782488488.773007:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300156288 00000100:00100000:2.0:1782488488.773071:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 639 00000100:00100000:2.0:1782488488.773083:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb545d18850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300156288:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488488.773155:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb545d18850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300156288:12345-0@lo:400:kworker.0 Request processed in 74us (1909us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488488.773163:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 639 00000100:00100000:2.0:1782488488.773310:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300156416 00000100:00100000:1.0:1782488488.773474:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45034ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300156288:0@lo:400:kworker.0 00000100:00100000:2.0:1782488488.773553:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300156416 00000100:00100000:2.0:1782488488.773575:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300156544 00000100:00100000:2.0:1782488488.773578:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300156544 00000100:00100000:2.0:1782488488.773592:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14116 00000100:00100000:2.0:1782488488.773597:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb450348380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300156416:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488488.773641:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488488.773678:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb450348380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300156416:12345-0@lo:38:kworker.0 Request processed in 79us (2332us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488488.773684:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14116 00000100:00100000:2.0:1782488488.773697:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14117 00000100:00100000:2.0:1782488488.773701:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45345b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300156544:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488488.773707:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488488.773730:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45345b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300156544:12345-0@lo:38:kworker.0 Request processed in 27us (936us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488488.773736:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14117 00000100:00080000:1.0:1782488488.774488:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488488.774497:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488488.774501:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45034a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300156416:0@lo:38:kworker.0 00000100:00080000:1.0:1782488488.774512:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488488.774515:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488488.774518:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb450348a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300156544:0@lo:38:kworker.0 00010000:00010000:2.0:1782488489.605564:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488490.625835:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488491.658535:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488492.682893:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488493.697737:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1782488493.890687:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488493.890695:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488493.890698:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488493.890735:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8116 00000100:00080000:1.0:1782488493.890740:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488493.890747:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488493.890770:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb45345bb80 x1869066300156800/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1782488493.890787:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488493.890790:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488493.890792:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488493.890796:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8116 00000100:00080000:1.0:1782488493.890798:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488493.890801:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488493.890812:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb453458e00 x1869066300156928/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1782488493.890841:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45345a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300156672:0@lo:400:kworker.0 00000100:00100000:1.0:1782488493.890929:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488493.890988:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45345bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300156800:0@lo:38:kworker.0 00000100:00100000:1.0:1782488493.891016:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488493.891034:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb453458e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300156928:0@lo:38:kworker.0 00000100:00100000:1.0:1782488493.891052:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488493.898816:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300156672 00000100:00100000:3.0:1782488493.898832:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300156672 00000100:00100000:3.0:1782488493.898899:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 640 00000100:00100000:3.0:1782488493.898909:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f342c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300156672:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782488493.899005:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f342c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300156672:12345-0@lo:400:kworker.0 Request processed in 98us (8076us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488493.899013:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 640 00000100:00100000:3.0:1782488493.899074:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45345a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300156672:0@lo:400:kworker.0 00000100:00100000:2.0:1782488493.899339:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300156800 00000100:00100000:2.0:1782488493.899344:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300156800 00000100:00100000:2.0:1782488493.899366:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300156928 00000100:00100000:2.0:1782488493.899369:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300156928 00000100:00100000:2.0:1782488493.899385:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14118 00000100:00100000:2.0:1782488493.899391:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb453459880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300156800:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488493.899405:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488493.906989:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14119 00000100:00100000:3.0:1782488493.907003:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb453458000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300156928:12345-0@lo:38:kworker.0 00010000:02000000:3.0:1782488493.907022:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488493.915713:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb453458000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300156928:12345-0@lo:38:kworker.0 Request processed in 8700us (24648us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488493.915741:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14119 00000100:00100000:2.0:1782488493.918338:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb453459880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300156800:12345-0@lo:38:kworker.0 Request processed in 18939us (27314us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488493.918354:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14118 00000100:00080000:1.0:1782488493.919514:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488493.919523:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488493.919527:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45345bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300156800:0@lo:38:kworker.0 00000100:00080000:1.0:1782488493.919540:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488493.919543:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488493.919593:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb453458e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300156928:0@lo:38:kworker.0 00010000:00010000:1.0:1782488494.726279:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488495.745754:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488496.772483:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488497.806914:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488498.820952:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1782488499.010932:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1782488499.010940:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1782488499.010957:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1782488499.010966:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8121 00000100:00080000:3.0:1782488499.010970:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1782488499.010977:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:3.0:1782488499.011002:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb455a2c380 x1869066300157184/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1782488499.011021:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1782488499.011024:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1782488499.011026:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1782488499.011030:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8121 00000100:00080000:3.0:1782488499.011032:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1782488499.011036:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:3.0:1782488499.011048:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb455a2ca80 x1869066300157312/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1782488499.011079:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb455a2f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300157056:0@lo:400:kworker.0 00000100:00100000:3.0:1782488499.011150:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488499.011211:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb455a2c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300157184:0@lo:38:kworker.0 00000100:00100000:3.0:1782488499.011237:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488499.011255:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb455a2ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300157312:0@lo:38:kworker.0 00000100:00100000:3.0:1782488499.011290:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488499.011323:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300157056 00000100:00100000:3.0:1782488499.011329:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300157056 00000100:00100000:3.0:1782488499.011370:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 641 00000100:00100000:3.0:1782488499.011378:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f343450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300157056:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782488499.011426:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f343450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300157056:12345-0@lo:400:kworker.0 Request processed in 49us (276us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488499.011441:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 641 00000100:00100000:3.0:1782488499.011473:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300157184 00000100:00100000:3.0:1782488499.011477:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300157184 00000100:00100000:3.0:1782488499.011495:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300157312 00000100:00100000:3.0:1782488499.011497:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300157312 00000100:00100000:3.0:1782488499.011513:0:213935:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14120 00000100:00100000:3.0:1782488499.011517:0:213935:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455a2f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300157184:12345-0@lo:38:kworker.0 00010000:02000000:3.0:1782488499.011531:0:213935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488499.011564:0:213935:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455a2f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300157184:12345-0@lo:38:kworker.0 Request processed in 45us (325us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488499.011570:0:213935:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14120 00000100:00100000:3.0:1782488499.011581:0:213935:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14121 00000100:00100000:3.0:1782488499.011585:0:213935:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455a2c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300157312:12345-0@lo:38:kworker.0 00010000:02000000:3.0:1782488499.011591:0:213935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488499.011641:0:213935:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455a2c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300157312:12345-0@lo:38:kworker.0 Request processed in 53us (349us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488499.011646:0:213935:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14121 00000100:00080000:3.0:1782488499.011691:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1782488499.011696:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:3.0:1782488499.011700:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb455a2c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300157184:0@lo:38:kworker.0 00000100:00080000:3.0:1782488499.011709:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1782488499.011712:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:3.0:1782488499.011715:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb455a2ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300157312:0@lo:38:kworker.0 00000100:00100000:3.0:1782488499.013870:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb455a2f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300157056:0@lo:400:kworker.0 00010000:00010000:1.0:1782488499.844917:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488500.867632:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488501.890897:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488502.914288:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488503.938350:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488504.133315:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488504.133321:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488504.133325:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488504.133333:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8126 00000100:00080000:0.0:1782488504.133338:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488504.133344:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488504.133363:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb579575880 x1869066300157568/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1782488504.133381:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488504.133385:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488504.133386:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488504.133389:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8126 00000100:00080000:0.0:1782488504.133392:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488504.133396:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488504.133439:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb468ff1180 x1869066300157696/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488504.133572:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1782488504.133588:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579576300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300157440:0@lo:400:kworker.0 00000100:00100000:0.0:1782488504.134075:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488504.134128:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579575880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300157568:0@lo:38:kworker.0 00000100:00100000:0.0:1782488504.134155:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488504.134193:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300157696:0@lo:38:kworker.0 00000100:00100000:0.0:1782488504.134214:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488504.134426:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300157440 00000100:00100000:0.0:1782488504.134434:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300157440 00000100:00100000:0.0:1782488504.134475:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 642 00000100:00100000:0.0:1782488504.134482:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f343c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300157440:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488504.134535:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f343c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300157440:12345-0@lo:400:kworker.0 Request processed in 53us (461us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488504.134542:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 642 00000100:00100000:0.0:1782488504.134789:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300157568 00000100:00100000:0.0:1782488504.134793:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300157568 00000100:00100000:0.0:1782488504.134811:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300157696 00000100:00100000:0.0:1782488504.134813:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300157696 00000100:00100000:0.0:1782488504.135982:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14122 00000100:00100000:0.0:1782488504.135990:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300157568:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488504.136006:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488504.136059:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300157568:12345-0@lo:38:kworker.0 Request processed in 68us (1901us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488504.136065:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14122 00000100:00100000:0.0:1782488504.136078:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14123 00000100:00100000:0.0:1782488504.136082:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb468ff2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300157696:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488504.136088:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488504.136107:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb468ff2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300157696:12345-0@lo:38:kworker.0 Request processed in 23us (1892us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488504.136112:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14123 00000100:00100000:0.0:1782488504.145546:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579576300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300157440:0@lo:400:kworker.0 00000100:00080000:0.0:1782488504.145634:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488504.145662:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488504.145666:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579575880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300157568:0@lo:38:kworker.0 00000100:00080000:0.0:1782488504.145679:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488504.145683:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488504.145686:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300157696:0@lo:38:kworker.0 00010000:00010000:1.0:1782488504.969054:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488505.988356:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488507.017964:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488508.040860:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488509.057715:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488509.249751:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488509.249758:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488509.249762:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488509.249771:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8131 00000100:00080000:0.0:1782488509.249775:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488509.249782:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488509.249817:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb579575880 x1869066300157952/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1782488509.249837:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488509.249841:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488509.249842:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488509.249845:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8131 00000100:00080000:0.0:1782488509.249848:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488509.249851:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488509.249864:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb579576300 x1869066300158080/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488509.249895:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579575880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300157952:0@lo:38:kworker.0 00000100:00100000:0.0:1782488509.249983:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488509.250041:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb579576300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300158080:0@lo:38:kworker.0 00000100:00100000:0.0:1782488509.250065:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488509.250315:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb468ff1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300157824:0@lo:400:kworker.0 00000100:00100000:3.0:1782488509.250390:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488509.250470:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300157952 00000100:00100000:3.0:1782488509.250479:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300157952 00000100:00100000:3.0:1782488509.250517:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300158080 00000100:00100000:3.0:1782488509.250521:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300158080 00000100:00100000:3.0:1782488509.250547:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14124 00000100:00100000:3.0:1782488509.250556:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb442932a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300157952:12345-0@lo:38:kworker.0 00010000:02000000:3.0:1782488509.250574:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488509.250642:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb442932a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300157952:12345-0@lo:38:kworker.0 Request processed in 85us (657us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488509.250651:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14124 00000100:00100000:3.0:1782488509.250675:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14125 00000100:00100000:3.0:1782488509.250681:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb442932300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300158080:12345-0@lo:38:kworker.0 00010000:02000000:3.0:1782488509.250690:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488509.250722:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb442932300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300158080:12345-0@lo:38:kworker.0 Request processed in 39us (655us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488509.250730:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14125 00000100:00100000:3.0:1782488509.250761:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300157824 00000100:00100000:3.0:1782488509.250765:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300157824 00000100:00100000:3.0:1782488509.250807:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 643 00000100:00100000:3.0:1782488509.250829:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f342050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300157824:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782488509.251988:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f342050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300157824:12345-0@lo:400:kworker.0 Request processed in 1169us (1594us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488509.252000:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 643 00000100:00100000:3.0:1782488509.252081:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb468ff1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300157824:0@lo:400:kworker.0 00000100:00080000:0.0:1782488509.254994:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488509.255007:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488509.255012:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579575880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300157952:0@lo:38:kworker.0 00000100:00080000:0.0:1782488509.255024:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488509.255027:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488509.255029:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb579576300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300158080:0@lo:38:kworker.0 00010000:00010000:0.0:1782488510.085138:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488511.106117:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488512.132197:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488513.153780:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488514.178917:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488514.374047:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488514.374054:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488514.374057:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488514.374065:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8136 00000100:00080000:0.0:1782488514.374069:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488514.374075:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488514.374094:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb442931500 x1869066300158336/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1782488514.374109:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488514.374114:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488514.374115:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488514.374119:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8136 00000100:00080000:0.0:1782488514.374121:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488514.374149:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488514.374159:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb442932d80 x1869066300158464/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488514.374186:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442931500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300158336:0@lo:38:kworker.0 00000100:00100000:0.0:1782488514.374248:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488514.374299:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442932d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300158464:0@lo:38:kworker.0 00000100:00100000:0.0:1782488514.374319:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488514.374358:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300158336 00000100:00100000:0.0:1782488514.374364:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300158336 00000100:00100000:0.0:1782488514.374391:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300158464 00000100:00100000:0.0:1782488514.374393:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300158464 00000100:00100000:0.0:1782488514.374408:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14126 00000100:00100000:0.0:1782488514.374414:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb442931880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300158336:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488514.374428:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488514.374498:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb442931880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300158336:12345-0@lo:38:kworker.0 Request processed in 82us (249us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488514.374505:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14126 00000100:00100000:0.0:1782488514.374520:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14127 00000100:00100000:0.0:1782488514.374524:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb442931180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300158464:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488514.374530:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488514.374547:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb442931180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300158464:12345-0@lo:38:kworker.0 Request processed in 22us (227us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488514.374552:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14127 00000100:00080000:0.0:1782488514.374582:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488514.374586:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488514.374590:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442931500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300158336:0@lo:38:kworker.0 00000100:00080000:0.0:1782488514.374638:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488514.374641:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488514.374644:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442932d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300158464:0@lo:38:kworker.0 00000100:00100000:3.0:1782488514.374861:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442931f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300158208:0@lo:400:kworker.0 00000100:00100000:3.0:1782488514.374921:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488514.374969:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300158208 00000100:00100000:3.0:1782488514.374972:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300158208 00000100:00100000:3.0:1782488514.375003:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 644 00000100:00100000:3.0:1782488514.375010:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f341850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300158208:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782488514.375048:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f341850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300158208:12345-0@lo:400:kworker.0 Request processed in 39us (126us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488514.375054:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 644 00000100:00100000:3.0:1782488514.375092:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442931f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300158208:0@lo:400:kworker.0 00010000:00010000:3.0:1782488515.201733:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488516.225740:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488517.258131:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488518.276802:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488519.297730:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1782488519.493861:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4bb706680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300158592:0@lo:400:kworker.0 00000100:00100000:1.0:1782488519.493937:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1782488519.494635:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488519.494642:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488519.494646:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488519.494653:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8141 00000100:00080000:2.0:1782488519.494657:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488519.494664:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488519.494691:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb4bb706300 x1869066300158720/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488519.494708:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488519.494712:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488519.494714:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488519.494717:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8141 00000100:00080000:2.0:1782488519.494720:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488519.494722:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488519.494731:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb4bb704700 x1869066300158848/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488519.494759:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4bb706300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300158720:0@lo:38:kworker.0 00000100:00100000:2.0:1782488519.494840:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488519.494861:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4bb704700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300158848:0@lo:38:kworker.0 00000100:00100000:2.0:1782488519.494878:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488519.494912:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300158720 00000100:00100000:2.0:1782488519.494919:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300158720 00000100:00100000:2.0:1782488519.494962:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300158848 00000100:00100000:2.0:1782488519.494965:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300158848 00000100:00100000:2.0:1782488519.494983:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14128 00000100:00100000:2.0:1782488519.494990:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4bb707100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300158720:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488519.495022:0:190296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488519.495059:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4bb707100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300158720:12345-0@lo:38:kworker.0 Request processed in 69us (218us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488519.495066:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14128 00000100:00100000:2.0:1782488519.495082:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14129 00000100:00100000:2.0:1782488519.495086:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4bb707480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300158848:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488519.495104:0:190296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488519.495124:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4bb707480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300158848:12345-0@lo:38:kworker.0 Request processed in 37us (245us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488519.495130:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14129 00000100:00080000:2.0:1782488519.495168:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488519.495172:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488519.495175:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4bb706300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300158720:0@lo:38:kworker.0 00000100:00080000:2.0:1782488519.495183:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488519.495186:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488519.495189:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4bb704700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300158848:0@lo:38:kworker.0 00000100:00100000:2.0:1782488519.495230:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300158592 00000100:00100000:2.0:1782488519.495234:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300158592 00000100:00100000:2.0:1782488519.495264:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 645 00000100:00100000:2.0:1782488519.495270:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f343050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300158592:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488519.498169:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f343050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300158592:12345-0@lo:400:kworker.0 Request processed in 2889us (4222us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488519.498185:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 645 00000100:00100000:1.0:1782488519.499799:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4bb706680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300158592:0@lo:400:kworker.0 00010000:00010000:0.0:1782488520.322116:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488521.346633:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488522.370602:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488523.393864:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488524.418089:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488524.609779:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488524.609798:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488524.609802:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488524.609810:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8146 00000100:00080000:0.0:1782488524.609814:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488524.609822:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488524.609845:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb442931500 x1869066300159104/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1782488524.609861:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488524.609865:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488524.609867:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488524.609870:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8146 00000100:00080000:0.0:1782488524.609872:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488524.609875:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488524.609883:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb442932680 x1869066300159232/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488524.609909:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442932d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300158976:0@lo:400:kworker.0 00000100:00100000:0.0:1782488524.610004:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488524.610081:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442931500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300159104:0@lo:38:kworker.0 00000100:00100000:0.0:1782488524.610136:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488524.610166:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442932680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300159232:0@lo:38:kworker.0 00000100:00100000:0.0:1782488524.610191:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488524.610227:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300158976 00000100:00100000:0.0:1782488524.610234:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300158976 00000100:00100000:0.0:1782488524.610273:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 646 00000100:00100000:0.0:1782488524.610280:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f342850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300158976:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488524.610325:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f342850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300158976:12345-0@lo:400:kworker.0 Request processed in 45us (321us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488524.610331:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 646 00000100:00100000:0.0:1782488524.610383:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300159104 00000100:00100000:0.0:1782488524.610386:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300159104 00000100:00100000:0.0:1782488524.610408:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300159232 00000100:00100000:0.0:1782488524.610410:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300159232 00000100:00100000:0.0:1782488524.610424:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14130 00000100:00100000:0.0:1782488524.610428:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb442930e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300159104:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488524.610450:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488524.610478:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb442930e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300159104:12345-0@lo:38:kworker.0 Request processed in 49us (341us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488524.610484:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14130 00000100:00100000:0.0:1782488524.610494:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14131 00000100:00100000:0.0:1782488524.610497:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb442931c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300159232:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488524.610503:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488524.610520:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb442931c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300159232:12345-0@lo:38:kworker.0 Request processed in 21us (328us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488524.610525:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14131 00000100:00080000:0.0:1782488524.610562:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488524.610583:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488524.610586:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442931500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300159104:0@lo:38:kworker.0 00000100:00080000:0.0:1782488524.610597:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488524.610600:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488524.610602:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442932680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300159232:0@lo:38:kworker.0 00000100:00100000:0.0:1782488524.610648:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442932d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300158976:0@lo:400:kworker.0 00010000:00010000:3.0:1782488525.444671:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488526.471268:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488527.490921:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488528.516897:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488529.543327:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1782488529.736908:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488529.736915:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488529.736920:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488529.736929:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8152 00000100:00080000:1.0:1782488529.736934:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488529.736940:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488529.736963:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb45345bb80 x1869066300159488/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1782488529.737034:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488529.737038:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488529.737040:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488529.737044:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8152 00000100:00080000:1.0:1782488529.737046:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488529.737049:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488529.737062:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb453458380 x1869066300159616/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1782488529.737086:0:3317:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1782488529.737118:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb453458e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300159360:0@lo:400:kworker.0 00000100:00100000:1.0:1782488529.737233:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488529.737305:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45345bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300159488:0@lo:38:kworker.0 00000100:00100000:1.0:1782488529.737359:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488529.737381:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb453458380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300159616:0@lo:38:kworker.0 00000100:00100000:1.0:1782488529.741330:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488529.741474:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300159360 00000100:00100000:1.0:1782488529.741484:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300159360 00000100:00100000:1.0:1782488529.741551:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 647 00000100:00100000:1.0:1782488529.741560:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f340050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300159360:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488529.741644:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f340050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300159360:12345-0@lo:400:kworker.0 Request processed in 85us (4411us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488529.741651:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 647 00000100:00100000:1.0:1782488529.741712:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300159488 00000100:00100000:1.0:1782488529.741717:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300159488 00000100:00100000:1.0:1782488529.741752:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300159616 00000100:00100000:1.0:1782488529.741755:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300159616 00000100:00100000:1.0:1782488529.741771:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14132 00000100:00100000:1.0:1782488529.741776:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb453458700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300159488:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488529.741791:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488529.741827:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb453458700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300159488:12345-0@lo:38:kworker.0 Request processed in 49us (4467us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488529.741833:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14132 00000100:00100000:1.0:1782488529.741845:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14133 00000100:00100000:1.0:1782488529.741849:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45345b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300159616:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488529.741856:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488529.741879:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45345b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300159616:12345-0@lo:38:kworker.0 Request processed in 28us (556us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488529.741885:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14133 00000100:00100000:1.0:1782488529.741932:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb453458e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300159360:0@lo:400:kworker.0 00000100:00080000:1.0:1782488529.741983:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488529.741989:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488529.741992:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45345bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300159488:0@lo:38:kworker.0 00000100:00080000:1.0:1782488529.742015:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488529.742019:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488529.742022:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb453458380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300159616:0@lo:38:kworker.0 00010000:00010000:2.0:1782488530.561783:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488531.589692:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488532.610561:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488533.637082:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488534.662346:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488534.853765:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488534.853773:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488534.853776:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488534.853785:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8157 00000100:00080000:0.0:1782488534.853789:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488534.853795:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488534.853816:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb442932680 x1869066300159872/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1782488534.853833:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488534.853836:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488534.853837:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488534.853841:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8157 00000100:00080000:0.0:1782488534.853843:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488534.853846:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488534.853857:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb442931500 x1869066300160000/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488534.853878:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1782488534.853906:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442932d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300159744:0@lo:400:kworker.0 00000100:00100000:0.0:1782488534.854016:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488534.854083:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442932680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300159872:0@lo:38:kworker.0 00000100:00100000:0.0:1782488534.854112:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488534.854131:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442931500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300160000:0@lo:38:kworker.0 00000100:00100000:0.0:1782488534.854149:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488534.854191:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300159744 00000100:00100000:0.0:1782488534.854199:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300159744 00000100:00100000:0.0:1782488534.854243:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 648 00000100:00100000:0.0:1782488534.854251:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f341450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300159744:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488534.854301:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f341450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300159744:12345-0@lo:400:kworker.0 Request processed in 50us (285us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488534.854307:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 648 00000100:00100000:0.0:1782488534.854342:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300159872 00000100:00100000:0.0:1782488534.854345:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300159872 00000100:00100000:0.0:1782488534.854378:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300160000 00000100:00100000:0.0:1782488534.854382:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300160000 00000100:00100000:0.0:1782488534.854400:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14134 00000100:00100000:0.0:1782488534.854405:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb442933b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300159872:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488534.854421:0:190296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488534.854454:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb442933b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300159872:12345-0@lo:38:kworker.0 Request processed in 47us (341us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488534.854461:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14134 00000100:00100000:0.0:1782488534.854473:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14135 00000100:00100000:0.0:1782488534.854476:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb442933100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300160000:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488534.854483:0:190296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488534.854509:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb442933100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300160000:12345-0@lo:38:kworker.0 Request processed in 31us (358us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488534.854522:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14135 00000100:00100000:2.0:1782488534.855053:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442932d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300159744:0@lo:400:kworker.0 00000100:00080000:2.0:1782488534.855091:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488534.855096:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488534.855099:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442932680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300159872:0@lo:38:kworker.0 00000100:00080000:2.0:1782488534.855108:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488534.855111:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488534.855113:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442931500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300160000:0@lo:38:kworker.0 00010000:00010000:0.0:1782488535.683719:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488536.705935:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488537.729748:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488538.753716:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488539.782625:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1782488539.971473:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488539.971481:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488539.971485:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488539.971505:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8162 00000100:00080000:1.0:1782488539.971510:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488539.971516:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488539.971540:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb45345bb80 x1869066300160256/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1782488539.971559:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488539.971563:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488539.971565:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488539.971568:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8162 00000100:00080000:1.0:1782488539.971570:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488539.971573:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488539.971584:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb453458e00 x1869066300160384/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1782488539.971604:0:3315:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1782488539.971636:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb453458380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300160128:0@lo:400:kworker.0 00000100:00100000:1.0:1782488539.971702:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488539.971750:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45345bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300160256:0@lo:38:kworker.0 00000100:00100000:1.0:1782488539.971775:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488539.971791:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb453458e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300160384:0@lo:38:kworker.0 00000100:00100000:1.0:1782488539.971807:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488539.971832:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300160128 00000100:00100000:1.0:1782488539.971839:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300160128 00000100:00100000:1.0:1782488539.971876:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 649 00000100:00100000:1.0:1782488539.971883:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f340850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300160128:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488539.971939:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f340850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300160128:12345-0@lo:400:kworker.0 Request processed in 55us (236us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488539.971951:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 649 00000100:00100000:1.0:1782488539.972022:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300160256 00000100:00100000:1.0:1782488539.972026:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300160256 00000100:00100000:1.0:1782488539.972058:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300160384 00000100:00100000:1.0:1782488539.972061:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300160384 00000100:00100000:1.0:1782488539.972077:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14136 00000100:00100000:1.0:1782488539.972082:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45345a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300160256:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488539.972096:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488539.972129:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45345a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300160256:12345-0@lo:38:kworker.0 Request processed in 45us (353us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488539.972135:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14136 00000100:00100000:1.0:1782488539.972147:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14137 00000100:00100000:1.0:1782488539.972151:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45345b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300160384:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488539.972158:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488539.972180:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45345b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300160384:12345-0@lo:38:kworker.0 Request processed in 27us (371us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488539.972185:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14137 00000100:00080000:1.0:1782488539.972224:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488539.972228:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488539.972231:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45345bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300160256:0@lo:38:kworker.0 00000100:00080000:1.0:1782488539.972240:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488539.972243:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488539.972245:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb453458e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300160384:0@lo:38:kworker.0 00000100:00100000:1.0:1782488539.972278:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb453458380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300160128:0@lo:400:kworker.0 00010000:00010000:3.0:1782488540.801771:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488541.825741:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488542.851584:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488543.874913:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488544.901173:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488545.090166:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488545.090172:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488545.090176:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488545.090183:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8167 00000100:00080000:0.0:1782488545.090187:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488545.090193:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488545.090213:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb442931500 x1869066300160640/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1782488545.090229:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488545.090232:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488545.090234:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488545.090237:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8167 00000100:00080000:0.0:1782488545.090240:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488545.090243:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488545.090251:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb442932680 x1869066300160768/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488545.090278:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442933480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300160512:0@lo:400:kworker.0 00000100:00100000:0.0:1782488545.090380:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488545.090437:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442931500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300160640:0@lo:38:kworker.0 00000100:00100000:0.0:1782488545.090474:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488545.090490:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442932680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300160768:0@lo:38:kworker.0 00000100:00100000:0.0:1782488545.090506:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488545.090533:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300160512 00000100:00100000:0.0:1782488545.090540:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300160512 00000100:00100000:0.0:1782488545.090577:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 650 00000100:00100000:0.0:1782488545.090585:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f342450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300160512:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488545.090644:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f342450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300160512:12345-0@lo:400:kworker.0 Request processed in 58us (263us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488545.090650:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 650 00000100:00100000:0.0:1782488545.090681:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300160640 00000100:00100000:0.0:1782488545.090685:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300160640 00000100:00100000:0.0:1782488545.090702:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300160768 00000100:00100000:0.0:1782488545.090704:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300160768 00000100:00100000:0.0:1782488545.090720:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14138 00000100:00100000:0.0:1782488545.090724:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb442932d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300160640:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488545.090738:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488545.090770:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb442932d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300160640:12345-0@lo:38:kworker.0 Request processed in 44us (294us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488545.090776:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14138 00000100:00100000:0.0:1782488545.090788:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14139 00000100:00100000:0.0:1782488545.090791:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb442931f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300160768:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488545.090797:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488545.090820:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb442931f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300160768:12345-0@lo:38:kworker.0 Request processed in 26us (312us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488545.090825:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14139 00000100:00080000:0.0:1782488545.090879:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488545.090890:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488545.090903:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442931500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300160640:0@lo:38:kworker.0 00000100:00080000:0.0:1782488545.090915:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488545.090918:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488545.090921:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442932680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300160768:0@lo:38:kworker.0 00000100:00100000:0.0:1782488545.090949:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442933480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300160512:0@lo:400:kworker.0 00010000:00010000:0.0:1782488545.923445:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488546.945888:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488547.972686:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488548.997085:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488550.019863:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1782488550.213726:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1782488550.213732:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1782488550.213736:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1782488550.213744:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8172 00000100:00080000:3.0:1782488550.213748:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1782488550.213755:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:3.0:1782488550.213775:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb455a2ca80 x1869066300161024/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1782488550.213790:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1782488550.213805:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1782488550.213807:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1782488550.213811:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8172 00000100:00080000:3.0:1782488550.213813:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1782488550.213817:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:3.0:1782488550.213839:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb455a2c380 x1869066300161152/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1782488550.213861:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb455a2f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300160896:0@lo:400:kworker.0 00000100:00100000:3.0:1782488550.213865:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb455a2ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300161024:0@lo:38:kworker.0 00000100:00100000:1.0:1782488550.217690:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488550.217772:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300160896 00000100:00100000:1.0:1782488550.217780:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300160896 00000100:00100000:1.0:1782488550.217835:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 651 00000100:00100000:1.0:1782488550.217845:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f341050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300160896:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488550.217893:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f341050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300160896:12345-0@lo:400:kworker.0 Request processed in 50us (208us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488550.217900:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 651 00000100:00100000:1.0:1782488550.217948:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb455a2f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300160896:0@lo:400:kworker.0 00000100:00100000:3.0:1782488550.223972:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488550.224030:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb455a2c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300161152:0@lo:38:kworker.0 00000100:00100000:3.0:1782488550.224065:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488550.224115:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300161024 00000100:00100000:3.0:1782488550.224122:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300161024 00000100:00100000:3.0:1782488550.224181:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300161152 00000100:00100000:3.0:1782488550.224185:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300161152 00000100:00100000:3.0:1782488550.224207:0:213935:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14140 00000100:00100000:3.0:1782488550.224213:0:213935:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455a2e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300161024:12345-0@lo:38:kworker.0 00010000:02000000:3.0:1782488550.224230:0:213935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488550.224270:0:213935:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455a2e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300161024:12345-0@lo:38:kworker.0 Request processed in 55us (305us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488550.224276:0:213935:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14140 00000100:00100000:3.0:1782488550.224290:0:213935:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14141 00000100:00100000:3.0:1782488550.224294:0:213935:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455a2e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300161152:12345-0@lo:38:kworker.0 00010000:02000000:3.0:1782488550.224301:0:213935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488550.224335:0:213935:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455a2e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300161152:12345-0@lo:38:kworker.0 Request processed in 40us (270us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488550.224357:0:213935:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14141 00000100:00080000:3.0:1782488550.224397:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1782488550.224403:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:3.0:1782488550.224407:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb455a2ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300161024:0@lo:38:kworker.0 00000100:00080000:3.0:1782488550.224415:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1782488550.224418:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:3.0:1782488550.224421:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb455a2c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300161152:0@lo:38:kworker.0 00010000:00010000:1.0:1782488551.041848:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488552.065727:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488553.091697:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488554.113729:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488555.138846:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1782488555.329831:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488555.329839:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488555.329844:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1782488555.329847:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb453458380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300161280:0@lo:400:kworker.0 00000100:00080000:1.0:1782488555.329853:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8177 00000100:00080000:1.0:1782488555.329860:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488555.329868:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488555.329893:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb453458e00 x1869066300161408/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1782488555.329913:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488555.329917:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488555.329920:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488555.329924:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8177 00000100:00080000:1.0:1782488555.329928:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1782488555.329935:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1782488555.329948:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488555.329965:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb45345bb80 x1869066300161536/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1782488555.330016:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb453458e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300161408:0@lo:38:kworker.0 00000100:00100000:3.0:1782488555.330051:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488555.330084:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45345bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300161536:0@lo:38:kworker.0 00000100:00100000:3.0:1782488555.330108:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488555.330162:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300161280 00000100:00100000:3.0:1782488555.330171:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300161280 00000100:00100000:3.0:1782488555.330223:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 652 00000100:00100000:3.0:1782488555.330233:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f79c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300161280:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782488555.330289:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f79c050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300161280:12345-0@lo:400:kworker.0 Request processed in 56us (356us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488555.330315:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 652 00000100:00100000:3.0:1782488555.330357:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300161408 00000100:00100000:3.0:1782488555.330362:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300161408 00000100:00100000:3.0:1782488555.330385:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300161536 00000100:00100000:3.0:1782488555.330388:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300161536 00000100:00100000:3.0:1782488555.330464:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14142 00000100:00100000:3.0:1782488555.330470:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455a2c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300161408:12345-0@lo:38:kworker.0 00010000:02000000:3.0:1782488555.330485:0:190296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488555.330525:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455a2c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300161408:12345-0@lo:38:kworker.0 Request processed in 53us (474us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488555.330533:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14142 00000100:00100000:3.0:1782488555.330560:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14143 00000100:00100000:3.0:1782488555.330566:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455a2ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300161536:12345-0@lo:38:kworker.0 00010000:02000000:3.0:1782488555.330575:0:190296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488555.330604:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455a2ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300161536:12345-0@lo:38:kworker.0 Request processed in 37us (494us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488555.330631:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14143 00000100:00080000:3.0:1782488555.331006:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1782488555.331013:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:3.0:1782488555.331018:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb453458e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300161408:0@lo:38:kworker.0 00000100:00080000:3.0:1782488555.331032:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1782488555.331037:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:3.0:1782488555.331042:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45345bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300161536:0@lo:38:kworker.0 00000100:00100000:3.0:1782488555.331082:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb453458380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300161280:0@lo:400:kworker.0 00010000:00010000:0.0:1782488556.161730:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488557.185826:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488558.210355:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488559.233917:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488560.258785:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1782488560.451111:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1782488560.451119:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1782488560.451123:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1782488560.451132:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8182 00000100:00080000:3.0:1782488560.451137:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1782488560.451154:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:3.0:1782488560.451266:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb455a2d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300161664:0@lo:400:kworker.0 00000100:00100000:3.0:1782488560.451334:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488560.451421:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300161664 00000100:00100000:3.0:1782488560.451428:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300161664 00000100:00100000:3.0:1782488560.451469:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 653 00000100:00100000:3.0:1782488560.451477:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f79c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300161664:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782488560.451522:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f79c450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300161664:12345-0@lo:400:kworker.0 Request processed in 46us (189us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488560.451528:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 653 00000100:00100000:3.0:1782488560.451573:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb455a2fb80 x1869066300161792/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:3.0:1782488560.451586:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1782488560.451590:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1782488560.451591:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1782488560.451595:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8182 00000100:00080000:3.0:1782488560.451597:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1782488560.451601:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:3.0:1782488560.451628:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb455a2ed80 x1869066300161920/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:3.0:1782488560.451649:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb455a2fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300161792:0@lo:38:kworker.0 00000100:00100000:3.0:1782488560.451701:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488560.451742:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb455a2ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300161920:0@lo:38:kworker.0 00000100:00100000:3.0:1782488560.451779:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488560.451823:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300161792 00000100:00100000:3.0:1782488560.451827:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300161792 00000100:00100000:3.0:1782488560.451858:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300161920 00000100:00100000:3.0:1782488560.451862:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300161920 00000100:00100000:3.0:1782488560.451880:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14144 00000100:00100000:3.0:1782488560.451885:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455a2dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300161792:12345-0@lo:38:kworker.0 00010000:02000000:3.0:1782488560.451897:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488560.451929:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455a2dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300161792:12345-0@lo:38:kworker.0 Request processed in 43us (228us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488560.451935:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14144 00000100:00100000:3.0:1782488560.451946:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14145 00000100:00100000:3.0:1782488560.451949:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455a2ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300161920:12345-0@lo:38:kworker.0 00010000:02000000:3.0:1782488560.451954:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488560.451972:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455a2ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300161920:12345-0@lo:38:kworker.0 Request processed in 22us (195us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488560.451977:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14145 00000100:00100000:3.0:1782488560.452019:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb455a2d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300161664:0@lo:400:kworker.0 00000100:00080000:3.0:1782488560.452048:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1782488560.452052:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:3.0:1782488560.452055:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb455a2fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300161792:0@lo:38:kworker.0 00000100:00080000:3.0:1782488560.452065:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1782488560.452068:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:3.0:1782488560.452072:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb455a2ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300161920:0@lo:38:kworker.0 00010000:00010000:1.0:1782488561.281764:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488562.309727:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488563.329758:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488564.355489:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488565.381927:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1782488565.569774:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488565.569782:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488565.569786:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488565.569794:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8187 00000100:00080000:1.0:1782488565.569799:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488565.569806:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488565.569829:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb453458380 x1869066300162176/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1782488565.570705:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488565.570710:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488565.570713:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488565.570718:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8187 00000100:00080000:1.0:1782488565.570721:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488565.570727:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488565.570746:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb45345bb80 x1869066300162304/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1782488565.570794:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb453458a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300162048:0@lo:400:kworker.0 00000100:00100000:1.0:1782488565.570862:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488565.570939:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb453458380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300162176:0@lo:38:kworker.0 00000100:00100000:1.0:1782488565.570984:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488565.571005:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45345bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300162304:0@lo:38:kworker.0 00000100:00100000:1.0:1782488565.571023:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488565.571054:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300162048 00000100:00100000:1.0:1782488565.571072:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300162048 00000100:00100000:1.0:1782488565.571113:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 654 00000100:00100000:1.0:1782488565.571122:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f79d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300162048:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488565.571181:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f79d850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300162048:12345-0@lo:400:kworker.0 Request processed in 59us (320us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488565.571190:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 654 00000100:00100000:1.0:1782488565.571233:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300162176 00000100:00100000:1.0:1782488565.571237:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300162176 00000100:00100000:1.0:1782488565.571260:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300162304 00000100:00100000:1.0:1782488565.571264:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300162304 00000100:00100000:1.0:1782488565.571285:0:213935:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14146 00000100:00100000:1.0:1782488565.571291:0:213935:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb453458e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300162176:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488565.571306:0:213935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488565.571346:0:213935:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb453458e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300162176:12345-0@lo:38:kworker.0 Request processed in 54us (360us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488565.571354:0:213935:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14146 00000100:00100000:1.0:1782488565.571385:0:213935:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14147 00000100:00100000:1.0:1782488565.571391:0:213935:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45345a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300162304:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488565.571400:0:213935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488565.571441:0:213935:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45345a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300162304:12345-0@lo:38:kworker.0 Request processed in 49us (416us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488565.571448:0:213935:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14147 00000100:00080000:1.0:1782488565.572294:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488565.572333:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488565.572337:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb453458380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300162176:0@lo:38:kworker.0 00000100:00080000:1.0:1782488565.572350:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488565.572354:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488565.572357:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45345bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300162304:0@lo:38:kworker.0 00000100:00100000:1.0:1782488565.572499:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb453458a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300162048:0@lo:400:kworker.0 00010000:00010000:0.0:1782488566.403264:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488567.428237:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488568.451715:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488569.474206:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488570.498851:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1782488570.689830:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488570.689841:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488570.689845:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488570.689857:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8192 00000100:00080000:1.0:1782488570.689861:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488570.689867:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488570.689894:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb45345bb80 x1869066300162560/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1782488570.689914:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488570.689918:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488570.689920:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488570.689924:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8192 00000100:00080000:1.0:1782488570.689927:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488570.689932:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488570.689943:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb453458380 x1869066300162688/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1782488570.689978:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45345bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300162560:0@lo:38:kworker.0 00000100:00100000:1.0:1782488570.690071:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488570.690132:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb453458380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300162688:0@lo:38:kworker.0 00000100:00100000:1.0:1782488570.690153:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488570.690262:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb453458a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300162432:0@lo:400:kworker.0 00000100:00100000:3.0:1782488570.690314:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488570.690400:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300162560 00000100:00100000:3.0:1782488570.690424:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300162560 00000100:00100000:3.0:1782488570.690479:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300162688 00000100:00100000:3.0:1782488570.690483:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300162688 00000100:00100000:3.0:1782488570.690506:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14148 00000100:00100000:3.0:1782488570.690512:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb450348a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300162560:12345-0@lo:38:kworker.0 00010000:02000000:3.0:1782488570.690535:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488570.690576:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb450348a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300162560:12345-0@lo:38:kworker.0 Request processed in 62us (504us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488570.690584:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14148 00000100:00100000:3.0:1782488570.690603:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14149 00000100:00100000:3.0:1782488570.690631:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45034a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300162688:12345-0@lo:38:kworker.0 00010000:02000000:3.0:1782488570.690639:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488570.690678:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45034a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300162688:12345-0@lo:38:kworker.0 Request processed in 47us (523us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488570.690683:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14149 00000100:00080000:3.0:1782488570.690740:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1782488570.690744:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:3.0:1782488570.690748:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45345bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300162560:0@lo:38:kworker.0 00000100:00080000:3.0:1782488570.690756:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1782488570.690760:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:3.0:1782488570.690762:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb453458380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300162688:0@lo:38:kworker.0 00000100:00100000:1.0:1782488570.690840:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300162432 00000100:00100000:1.0:1782488570.690844:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300162432 00000100:00100000:1.0:1782488570.690908:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 655 00000100:00100000:1.0:1782488570.690918:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f79d450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300162432:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1782488570.690971:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f79d450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300162432:12345-0@lo:400:kworker.0 Request processed in 55us (655us total) trans 0 rc 0/0 00000100:00100000:1.0:1782488570.690978:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 655 00000100:00100000:1.0:1782488570.691039:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb453458a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300162432:0@lo:400:kworker.0 00010000:00010000:0.0:1782488571.521824:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488572.546661:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488573.570163:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488574.593813:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488575.622847:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1782488575.810958:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488575.810967:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488575.810972:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488575.810984:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8198 00000100:00100000:3.0:1782488575.810987:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb45034ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300162816:0@lo:400:kworker.0 00000100:00080000:1.0:1782488575.810990:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488575.811002:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:3.0:1782488575.811084:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488575.811293:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300162816 00000100:00100000:3.0:1782488575.811309:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300162816 00000100:00100000:3.0:1782488575.811403:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 656 00000100:00100000:3.0:1782488575.811443:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f79f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300162816:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782488575.811559:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f79f450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300162816:12345-0@lo:400:kworker.0 Request processed in 121us (474us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488575.811570:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 656 00000100:00100000:3.0:1782488575.811929:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb45034ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300162816:0@lo:400:kworker.0 00000100:00100000:1.0:1782488575.812722:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb4cf55ad80 x1869066300162944/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1782488575.812755:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488575.812761:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488575.812764:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488575.812768:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8198 00000100:00080000:1.0:1782488575.812781:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488575.812787:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488575.812805:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb4cf558380 x1869066300163072/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1782488575.812839:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4cf55ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300162944:0@lo:38:kworker.0 00000100:00100000:1.0:1782488575.812895:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488575.812962:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4cf558380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300163072:0@lo:38:kworker.0 00000100:00100000:1.0:1782488575.812994:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1782488575.813057:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300162944 00000100:00100000:1.0:1782488575.813075:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300162944 00000100:00100000:1.0:1782488575.813106:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300163072 00000100:00100000:1.0:1782488575.813109:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300163072 00000100:00100000:1.0:1782488575.813126:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14150 00000100:00100000:1.0:1782488575.813133:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4cf55b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300162944:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488575.813150:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488575.813539:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4cf55b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300162944:12345-0@lo:38:kworker.0 Request processed in 324us (563us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488575.813549:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14150 00000100:00100000:1.0:1782488575.816733:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14151 00000100:00100000:1.0:1782488575.816749:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4cf55a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300163072:12345-0@lo:38:kworker.0 00010000:02000000:1.0:1782488575.816770:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00080000:1.0:1782488575.820659:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488575.820676:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488575.820684:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4cf55ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300162944:0@lo:38:kworker.0 00000100:00100000:1.0:1782488575.820823:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4cf55a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300163072:12345-0@lo:38:kworker.0 Request processed in 4072us (7825us total) trans 0 rc -19/-19 00000100:00100000:1.0:1782488575.820855:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14151 00000100:00080000:1.0:1782488575.820925:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1782488575.821795:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:1.0:1782488575.821804:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4cf558380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300163072:0@lo:38:kworker.0 00010000:00010000:3.0:1782488576.650164:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488577.665777:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488578.690701:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488579.714629:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488580.737740:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488580.929747:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488580.929753:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488580.929757:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488580.929764:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8203 00000100:00080000:0.0:1782488580.929780:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488580.929787:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488580.929811:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb442932680 x1869066300163328/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1782488580.929829:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488580.929832:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488580.929834:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488580.929837:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8203 00000100:00080000:0.0:1782488580.929840:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488580.929843:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488580.929857:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb442931500 x1869066300163456/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488580.929885:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442933480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300163200:0@lo:400:kworker.0 00000100:00100000:3.0:1782488580.930248:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442932680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300163328:0@lo:38:kworker.0 00000100:00100000:3.0:1782488580.930326:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488580.930368:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442931500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300163456:0@lo:38:kworker.0 00000100:00100000:3.0:1782488580.930388:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488580.930431:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300163328 00000100:00100000:3.0:1782488580.930438:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300163328 00000100:00100000:3.0:1782488580.930482:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300163456 00000100:00100000:3.0:1782488580.930485:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300163456 00000100:00100000:3.0:1782488580.930523:0:213935:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14152 00000100:00100000:3.0:1782488580.930531:0:213935:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455a2ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300163328:12345-0@lo:38:kworker.0 00010000:02000000:3.0:1782488580.930548:0:213935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488580.930630:0:213935:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455a2ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300163328:12345-0@lo:38:kworker.0 Request processed in 75us (280us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488580.930638:0:213935:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14152 00000100:00100000:3.0:1782488580.930662:0:213935:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14153 00000100:00100000:3.0:1782488580.930667:0:213935:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455a2fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300163456:12345-0@lo:38:kworker.0 00010000:02000000:3.0:1782488580.930674:0:213935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488580.930697:0:213935:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455a2fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300163456:12345-0@lo:38:kworker.0 Request processed in 30us (308us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488580.930720:0:213935:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14153 00000100:00080000:3.0:1782488580.930768:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1782488580.930772:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:3.0:1782488580.930776:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442932680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300163328:0@lo:38:kworker.0 00000100:00080000:3.0:1782488580.930785:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1782488580.930788:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:3.0:1782488580.930791:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442931500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300163456:0@lo:38:kworker.0 00000100:00100000:0.0:1782488580.932376:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1782488580.932463:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300163200 00000100:00100000:3.0:1782488580.932474:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300163200 00000100:00100000:3.0:1782488580.932538:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 657 00000100:00100000:3.0:1782488580.932549:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f79fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300163200:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782488580.932649:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f79fc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300163200:12345-0@lo:400:kworker.0 Request processed in 102us (283us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488580.932659:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 657 00000100:00100000:0.0:1782488580.932709:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442933480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300163200:0@lo:400:kworker.0 00010000:00010000:0.0:1782488581.761731:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488582.785773:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488583.809743:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488584.833756:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488585.857733:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488586.049747:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488586.049755:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488586.049768:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488586.049778:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8208 00000100:00080000:0.0:1782488586.049782:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488586.049803:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488586.049836:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb442931500 x1869066300163712/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1782488586.049861:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488586.049865:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488586.049868:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488586.049873:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8208 00000100:00080000:0.0:1782488586.049876:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488586.049888:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488586.049906:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb442932680 x1869066300163840/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488586.049939:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442933480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300163584:0@lo:400:kworker.0 00000100:00100000:0.0:1782488586.050030:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488586.050119:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442931500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300163712:0@lo:38:kworker.0 00000100:00100000:0.0:1782488586.050157:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488586.050202:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb442932680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300163840:0@lo:38:kworker.0 00000100:00100000:0.0:1782488586.050234:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488586.050387:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300163584 00000100:00100000:0.0:1782488586.050403:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300163584 00000100:00100000:2.0:1782488586.050438:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300163712 00000100:00100000:2.0:1782488586.050472:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300163712 00000100:00100000:0.0:1782488586.050526:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 658 00000100:00100000:0.0:1782488586.050546:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f79cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300163584:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488586.050679:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f79cc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300163584:12345-0@lo:400:kworker.0 Request processed in 139us (647us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488586.050688:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 658 00000100:00100000:2.0:1782488586.050696:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300163840 00000100:00100000:2.0:1782488586.050703:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300163840 00000100:00100000:2.0:1782488586.050741:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14154 00000100:00100000:2.0:1782488586.050754:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579576300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300163712:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488586.050775:0:190296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488586.050796:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14155 00000100:00100000:0.0:1782488586.050809:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579575880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300163840:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488586.050827:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488586.050844:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579576300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300163712:12345-0@lo:38:kworker.0 Request processed in 91us (684us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488586.050855:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14154 00000100:00100000:0.0:1782488586.050887:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579575880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300163840:12345-0@lo:38:kworker.0 Request processed in 76us (649us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488586.050900:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14155 00000100:00080000:0.0:1782488586.051008:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488586.051017:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488586.051022:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442931500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300163712:0@lo:38:kworker.0 00000100:00080000:0.0:1782488586.051047:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488586.051054:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488586.051062:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442932680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300163840:0@lo:38:kworker.0 00000100:00100000:2.0:1782488586.051144:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb442933480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300163584:0@lo:400:kworker.0 00010000:00010000:1.0:1782488586.881726:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488587.906540:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488588.929749:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488589.953869:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488590.977855:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488591.169816:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488591.169829:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488591.169837:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488591.169860:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8213 00000100:00080000:2.0:1782488591.169864:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488591.169871:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488591.169906:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb4bb706300 x1869066300164096/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488591.169918:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488591.169921:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488591.169922:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488591.169925:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8213 00000100:00080000:2.0:1782488591.169927:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488591.169930:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488591.169939:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb4bb704000 x1869066300164224/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488591.169967:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4bb704700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300163968:0@lo:400:kworker.0 00000100:00100000:2.0:1782488591.170084:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488591.170166:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300163968 00000100:00100000:2.0:1782488591.170174:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300163968 00000100:00100000:2.0:1782488591.170227:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 659 00000100:00100000:2.0:1782488591.170235:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f79dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300163968:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488591.170276:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f79dc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300163968:12345-0@lo:400:kworker.0 Request processed in 42us (197us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488591.170282:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 659 00000100:00100000:2.0:1782488591.170320:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4bb704700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300163968:0@lo:400:kworker.0 00000100:00100000:0.0:1782488591.170411:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4bb706300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300164096:0@lo:38:kworker.0 00000100:00100000:0.0:1782488591.170480:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488591.170525:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4bb704000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300164224:0@lo:38:kworker.0 00000100:00100000:0.0:1782488591.170555:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488591.170654:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300164096 00000100:00100000:0.0:1782488591.170659:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300164096 00000100:00100000:0.0:1782488591.170704:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300164224 00000100:00100000:0.0:1782488591.170708:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300164224 00000100:00100000:0.0:1782488591.170731:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14156 00000100:00100000:0.0:1782488591.170737:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb442933480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300164096:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488591.170763:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488591.170802:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb442933480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300164096:12345-0@lo:38:kworker.0 Request processed in 63us (322us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488591.170818:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14156 00000100:00100000:0.0:1782488591.170835:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14157 00000100:00100000:0.0:1782488591.170841:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb442932680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300164224:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488591.170849:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488591.170880:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb442932680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300164224:12345-0@lo:38:kworker.0 Request processed in 38us (325us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488591.170885:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14157 00000100:00080000:0.0:1782488591.170925:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488591.170929:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488591.170932:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4bb706300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300164096:0@lo:38:kworker.0 00000100:00080000:0.0:1782488591.170941:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488591.170943:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488591.170946:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4bb704000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300164224:0@lo:38:kworker.0 00010000:00010000:0.0:1782488592.001755:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488593.025826:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488594.049801:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488595.073799:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488596.097731:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488596.289766:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488596.289781:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488596.289784:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488596.289792:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8218 00000100:00080000:2.0:1782488596.289795:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488596.289801:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488596.289820:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb4bb706d80 x1869066300164480/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488596.289835:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488596.289839:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488596.289840:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488596.289844:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8218 00000100:00080000:2.0:1782488596.289846:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488596.289849:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488596.289858:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb4bb706a00 x1869066300164608/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488596.289883:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4bb704700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300164352:0@lo:400:kworker.0 00000100:00100000:2.0:1782488596.289979:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488596.290106:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300164352 00000100:00100000:2.0:1782488596.290113:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300164352 00000100:00100000:2.0:1782488596.290181:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 660 00000100:00100000:2.0:1782488596.290192:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f79c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300164352:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488596.290264:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f79c850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300164352:12345-0@lo:400:kworker.0 Request processed in 74us (285us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488596.290272:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 660 00000100:00100000:2.0:1782488596.290347:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4bb704700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300164352:0@lo:400:kworker.0 00000100:00100000:0.0:1782488596.290385:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4bb706d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300164480:0@lo:38:kworker.0 00000100:00100000:0.0:1782488596.290466:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488596.290505:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300164480 00000100:00100000:2.0:1782488596.290511:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300164480 00000100:00100000:0.0:1782488596.290514:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4bb706a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300164608:0@lo:38:kworker.0 00000100:00100000:0.0:1782488596.290545:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488596.290564:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300164608 00000100:00100000:2.0:1782488596.290569:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300164608 00000100:00100000:2.0:1782488596.290599:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14158 00000100:00100000:2.0:1782488596.290629:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb442931500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300164480:12345-0@lo:38:kworker.0 00000100:00100000:0.0:1782488596.290631:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14159 00000100:00100000:0.0:1782488596.290641:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb545d93b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300164608:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488596.290650:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00010000:02000000:0.0:1782488596.290653:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488596.290701:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb442931500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300164480:12345-0@lo:38:kworker.0 Request processed in 94us (255us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488596.290706:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb545d93b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300164608:12345-0@lo:38:kworker.0 Request processed in 62us (157us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488596.290710:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14158 00000100:00100000:0.0:1782488596.290720:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14159 00000100:00080000:0.0:1782488596.290771:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488596.290777:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488596.290781:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4bb706d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300164480:0@lo:38:kworker.0 00000100:00080000:0.0:1782488596.290795:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488596.290799:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488596.290802:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4bb706a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300164608:0@lo:38:kworker.0 00010000:00010000:0.0:1782488597.121752:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488598.145739:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488599.169737:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488600.193727:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488601.219021:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488601.409715:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488601.409721:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488601.409724:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488601.409731:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8223 00000100:00080000:0.0:1782488601.409734:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00100000:2.0:1782488601.409740:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb545d92a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300164736:0@lo:400:kworker.0 00000100:00080000:0.0:1782488601.409741:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488601.409764:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb545d90a80 x1869066300164864/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1782488601.409786:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488601.409790:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488601.409793:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488601.409798:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8223 00000100:00100000:2.0:1782488601.409805:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1782488601.409821:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488601.409826:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488601.409838:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb545d91c00 x1869066300164992/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488601.409865:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb545d90a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300164864:0@lo:38:kworker.0 00000100:00100000:0.0:1782488601.409905:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488601.409908:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300164736 00000100:00100000:2.0:1782488601.409916:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300164736 00000100:00100000:0.0:1782488601.409931:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb545d91c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300164992:0@lo:38:kworker.0 00000100:00100000:2.0:1782488601.409957:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 661 00000100:00100000:0.0:1782488601.409964:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488601.409966:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f79ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300164736:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488601.410018:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f79ec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300164736:12345-0@lo:400:kworker.0 Request processed in 54us (214us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488601.410020:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300164864 00000100:00100000:0.0:1782488601.410027:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300164864 00000100:00100000:2.0:1782488601.410031:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 661 00000100:00100000:0.0:1782488601.410072:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300164992 00000100:00100000:0.0:1782488601.410076:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300164992 00000100:00100000:2.0:1782488601.410091:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14160 00000100:00100000:2.0:1782488601.410100:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb545d90700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300164864:12345-0@lo:38:kworker.0 00000100:00100000:0.0:1782488601.410107:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14161 00010000:02000000:2.0:1782488601.410117:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488601.410124:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb451b29500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300164992:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488601.410135:0:190296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488601.410167:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb545d90700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300164864:12345-0@lo:38:kworker.0 Request processed in 65us (261us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488601.410179:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14160 00000100:00100000:0.0:1782488601.410180:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb451b29500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300164992:12345-0@lo:38:kworker.0 Request processed in 61us (214us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488601.410197:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14161 00000100:00100000:2.0:1782488601.410229:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb545d92a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300164736:0@lo:400:kworker.0 00000100:00080000:2.0:1782488601.410280:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488601.410286:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488601.410291:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb545d90a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300164864:0@lo:38:kworker.0 00000100:00080000:2.0:1782488601.410307:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488601.410311:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488601.410314:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb545d91c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300164992:0@lo:38:kworker.0 00010000:00010000:0.0:1782488602.241744:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488603.265753:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488604.289748:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488605.313921:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488606.337728:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488606.530040:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488606.530047:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488606.530052:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488606.530061:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8228 00000100:00080000:2.0:1782488606.530065:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488606.530071:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488606.530095:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb4bb705880 x1869066300165248/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488606.530117:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488606.530121:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488606.530123:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488606.530127:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8228 00000100:00080000:2.0:1782488606.530130:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488606.530134:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488606.530147:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb4bb704a80 x1869066300165376/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488606.530178:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4bb704700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300165120:0@lo:400:kworker.0 00000100:00100000:2.0:1782488606.530273:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488606.530360:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4bb705880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300165248:0@lo:38:kworker.0 00000100:00100000:2.0:1782488606.530388:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488606.530404:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4bb704a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300165376:0@lo:38:kworker.0 00000100:00100000:2.0:1782488606.530420:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488606.530464:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300165248 00000100:00100000:2.0:1782488606.530472:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300165248 00000100:00100000:2.0:1782488606.530511:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300165376 00000100:00100000:2.0:1782488606.530514:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300165376 00000100:00100000:2.0:1782488606.530531:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14162 00000100:00100000:2.0:1782488606.530537:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4bb704380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300165248:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488606.530550:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488606.530586:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4bb704380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300165248:12345-0@lo:38:kworker.0 Request processed in 48us (197us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488606.530591:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14162 00000100:00100000:2.0:1782488606.530623:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14163 00000100:00100000:2.0:1782488606.530626:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4bb705f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300165376:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488606.530632:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488606.530650:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4bb705f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300165376:12345-0@lo:38:kworker.0 Request processed in 22us (230us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488606.530654:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14163 00000100:00080000:2.0:1782488606.530692:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488606.530696:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488606.530699:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4bb705880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300165248:0@lo:38:kworker.0 00000100:00080000:2.0:1782488606.530709:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488606.530713:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488606.530722:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4bb704a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300165376:0@lo:38:kworker.0 00000100:00100000:2.0:1782488606.530934:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300165120 00000100:00100000:2.0:1782488606.530938:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300165120 00000100:00100000:2.0:1782488606.531018:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 662 00000100:00100000:2.0:1782488606.531028:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f79e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300165120:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488606.531074:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f79e450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300165120:12345-0@lo:400:kworker.0 Request processed in 48us (802us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488606.531101:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 662 00000100:00100000:2.0:1782488606.531149:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4bb704700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300165120:0@lo:400:kworker.0 00010000:00010000:0.0:1782488607.361751:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488608.385740:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488609.409718:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488610.434713:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488611.458695:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488611.650754:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488611.650761:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488611.650769:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488611.650778:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8233 00000100:00080000:0.0:1782488611.650782:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488611.650790:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488611.650813:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb451b28700 x1869066300165632/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1782488611.650833:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488611.650837:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488611.650840:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488611.650845:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8233 00000100:00080000:0.0:1782488611.650847:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488611.650851:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488611.650865:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb451b29180 x1869066300165760/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488611.650897:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb451b2bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300165504:0@lo:400:kworker.0 00000100:00100000:0.0:1782488611.650968:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488611.651048:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb451b28700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300165632:0@lo:38:kworker.0 00000100:00100000:0.0:1782488611.651086:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488611.651126:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb451b29180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300165760:0@lo:38:kworker.0 00000100:00100000:0.0:1782488611.651150:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488611.651190:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300165632 00000100:00100000:0.0:1782488611.651198:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300165632 00000100:00100000:0.0:1782488611.651232:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300165760 00000100:00100000:0.0:1782488611.651248:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300165760 00000100:00100000:0.0:1782488611.651274:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14164 00000100:00100000:0.0:1782488611.651282:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb451b2b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300165632:12345-0@lo:38:kworker.0 00000100:00100000:3.0:1782488611.651305:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300165504 00010000:02000000:0.0:1782488611.651309:0:190296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488611.651324:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300165504 00000100:00100000:0.0:1782488611.651359:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb451b2b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300165632:12345-0@lo:38:kworker.0 Request processed in 74us (270us total) trans 0 rc -19/-19 00000100:00100000:3.0:1782488611.651367:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 663 00000100:00100000:0.0:1782488611.651370:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14164 00000100:00100000:0.0:1782488611.651394:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14165 00000100:00100000:0.0:1782488611.651402:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb451b2b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300165760:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488611.651409:0:190296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1782488611.651436:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57f79e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300165504:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1782488611.651506:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57f79e050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300165504:12345-0@lo:400:kworker.0 Request processed in 128us (537us total) trans 0 rc 0/0 00000100:00100000:3.0:1782488611.651514:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 663 00000100:00080000:0.0:1782488611.651525:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488611.651530:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488611.651534:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb451b28700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300165632:0@lo:38:kworker.0 00000100:00100000:0.0:1782488611.651563:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb451b2bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300165504:0@lo:400:kworker.0 00000100:00100000:0.0:1782488611.651626:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb451b2b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300165760:12345-0@lo:38:kworker.0 Request processed in 205us (456us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488611.651631:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14165 00000100:00080000:0.0:1782488611.651654:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488611.651658:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488611.651661:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb451b29180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300165760:0@lo:38:kworker.0 00010000:00010000:0.0:1782488612.482702:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488613.505710:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488614.530689:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488615.554724:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488616.578703:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488616.769774:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488616.769782:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488616.769786:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488616.769796:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8239 00000100:00080000:2.0:1782488616.769800:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488616.769806:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488616.769830:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb4bb704a80 x1869066300166016/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488616.769844:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488616.769847:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488616.769849:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488616.769852:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8239 00000100:00080000:2.0:1782488616.769855:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488616.769858:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488616.769869:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb4bb705880 x1869066300166144/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488616.769952:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1782488616.769967:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4bb704700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300165888:0@lo:400:kworker.0 00000100:00100000:2.0:1782488616.770025:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488616.770081:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300165888 00000100:00100000:2.0:1782488616.770087:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300165888 00000100:00100000:0.0:1782488616.770103:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4bb704a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300166016:0@lo:38:kworker.0 00000100:00100000:2.0:1782488616.770130:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 664 00000100:00100000:2.0:1782488616.770142:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455218050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300165888:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488616.770160:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488616.770191:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4bb705880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300166144:0@lo:38:kworker.0 00000100:00100000:2.0:1782488616.770208:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455218050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300165888:12345-0@lo:400:kworker.0 Request processed in 67us (181us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488616.770228:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488616.770235:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 664 00000100:00100000:0.0:1782488616.770282:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300166016 00000100:00100000:0.0:1782488616.770286:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300166016 00000100:00100000:2.0:1782488616.770295:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300166144 00000100:00100000:2.0:1782488616.770299:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300166144 00000100:00100000:0.0:1782488616.770314:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14166 00000100:00100000:0.0:1782488616.770320:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb451b29180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300166016:12345-0@lo:38:kworker.0 00000100:00100000:2.0:1782488616.770327:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14167 00010000:02000000:0.0:1782488616.770333:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488616.770335:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb451b2bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300166144:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488616.770344:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488616.770374:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb451b29180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300166016:12345-0@lo:38:kworker.0 Request processed in 52us (214us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488616.770383:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14166 00000100:00100000:2.0:1782488616.770384:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb451b2bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300166144:12345-0@lo:38:kworker.0 Request processed in 47us (157us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488616.770390:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14167 00000100:00100000:2.0:1782488616.770422:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4bb704700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300165888:0@lo:400:kworker.0 00000100:00080000:0.0:1782488616.770454:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488616.770459:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488616.770462:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4bb704a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300166016:0@lo:38:kworker.0 00000100:00080000:0.0:1782488616.770472:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488616.770475:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488616.770478:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4bb705880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300166144:0@lo:38:kworker.0 00010000:00010000:0.0:1782488617.601744:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488618.625733:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488619.649742:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488620.673762:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488621.697728:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488621.889774:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488621.889782:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488621.889787:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488621.889796:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8244 00000100:00080000:0.0:1782488621.889845:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488621.889853:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488621.889878:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb451b2b100 x1869066300166400/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1782488621.889897:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488621.889902:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488621.889904:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488621.889908:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8244 00000100:00080000:0.0:1782488621.889912:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488621.889916:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488621.889931:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb451b2aa00 x1869066300166528/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488621.889964:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb451b28700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300166272:0@lo:400:kworker.0 00000100:00100000:0.0:1782488621.890035:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488621.890109:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb451b2b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300166400:0@lo:38:kworker.0 00000100:00100000:0.0:1782488621.890150:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488621.890177:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb451b2aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300166528:0@lo:38:kworker.0 00000100:00100000:0.0:1782488621.890243:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488621.890291:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300166272 00000100:00100000:0.0:1782488621.890299:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300166272 00000100:00100000:0.0:1782488621.890353:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 665 00000100:00100000:0.0:1782488621.890365:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45521a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300166272:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488621.890430:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300166400 00000100:00100000:0.0:1782488621.890432:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45521a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300166272:12345-0@lo:400:kworker.0 Request processed in 69us (396us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488621.890437:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300166400 00000100:00100000:0.0:1782488621.890444:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 665 00000100:00100000:2.0:1782488621.890464:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300166528 00000100:00100000:2.0:1782488621.890467:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300166528 00000100:00100000:2.0:1782488621.890489:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14168 00000100:00100000:2.0:1782488621.890495:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb44ed0b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300166400:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488621.890515:0:190296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488621.890523:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14169 00000100:00100000:0.0:1782488621.890529:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb44ed0aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300166528:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488621.890541:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488621.890572:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb44ed0b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300166400:12345-0@lo:38:kworker.0 Request processed in 76us (423us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488621.890579:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14168 00000100:00100000:0.0:1782488621.890596:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb44ed0aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300166528:12345-0@lo:38:kworker.0 Request processed in 64us (352us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488621.890628:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14169 00000100:00100000:0.0:1782488621.890677:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb451b28700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300166272:0@lo:400:kworker.0 00000100:00080000:0.0:1782488621.890719:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488621.890725:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488621.890729:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb451b2b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300166400:0@lo:38:kworker.0 00000100:00080000:0.0:1782488621.890743:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488621.890747:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488621.890751:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb451b2aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300166528:0@lo:38:kworker.0 00010000:00010000:0.0:1782488622.721710:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488623.745812:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488624.769766:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488625.793810:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488626.817866:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488627.009793:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488627.009800:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488627.009804:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488627.009813:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8249 00000100:00080000:0.0:1782488627.009817:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488627.009825:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488627.009859:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb44ed08000 x1869066300166784/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1782488627.009916:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488627.009919:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488627.009922:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488627.009926:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8249 00000100:00080000:0.0:1782488627.009929:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488627.009934:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488627.009951:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb44ed0a680 x1869066300166912/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488627.009973:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1782488627.010002:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb44ed08380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300166656:0@lo:400:kworker.0 00000100:00100000:2.0:1782488627.010403:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb44ed08000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300166784:0@lo:38:kworker.0 00000100:00100000:2.0:1782488627.010505:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488627.010580:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb44ed0a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300166912:0@lo:38:kworker.0 00000100:00100000:2.0:1782488627.010763:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488627.010937:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300166784 00000100:00100000:2.0:1782488627.010945:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300166784 00000100:00100000:2.0:1782488627.010996:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300166912 00000100:00100000:2.0:1782488627.011000:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300166912 00000100:00100000:2.0:1782488627.011023:0:213935:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14170 00000100:00100000:2.0:1782488627.011031:0:213935:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4bb704700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300166784:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488627.011059:0:213935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488627.011107:0:213935:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4bb704700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300166784:12345-0@lo:38:kworker.0 Request processed in 74us (603us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488627.011115:0:213935:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14170 00000100:00100000:2.0:1782488627.011139:0:213935:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14171 00000100:00100000:2.0:1782488627.011145:0:213935:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4bb705880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300166912:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488627.011153:0:213935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488627.011206:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488627.011214:0:213935:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4bb705880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300166912:12345-0@lo:38:kworker.0 Request processed in 66us (450us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488627.011222:0:213935:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14171 00000100:00100000:0.0:1782488627.011273:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300166656 00000100:00100000:0.0:1782488627.011279:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300166656 00000100:00080000:2.0:1782488627.011280:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488627.011288:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488627.011292:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb44ed08000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300166784:0@lo:38:kworker.0 00000100:00080000:2.0:1782488627.011309:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488627.011324:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488627.011328:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb44ed0a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300166912:0@lo:38:kworker.0 00000100:00100000:0.0:1782488627.011340:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 666 00000100:00100000:0.0:1782488627.011351:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455219050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300166656:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488627.011400:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455219050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300166656:12345-0@lo:400:kworker.0 Request processed in 51us (195us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488627.011408:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 666 00000100:00100000:2.0:1782488627.011432:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb44ed08380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300166656:0@lo:400:kworker.0 00010000:00010000:0.0:1782488627.842056:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488628.865772:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488629.889770:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488630.914356:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488631.937731:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488632.129770:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488632.129776:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488632.131019:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488632.131030:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8254 00000100:00080000:0.0:1782488632.131035:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488632.131043:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488632.131073:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb44ed0ad80 x1869066300167168/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1782488632.131092:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488632.131095:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488632.131096:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488632.131100:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8254 00000100:00080000:0.0:1782488632.131102:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488632.131106:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488632.131133:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb44ed09f80 x1869066300167296/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488632.131169:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb44ed09500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300167040:0@lo:400:kworker.0 00000100:00100000:0.0:1782488632.131268:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488632.131279:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb44ed0ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300167168:0@lo:38:kworker.0 00000100:00100000:2.0:1782488632.131324:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488632.131347:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb44ed09f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300167296:0@lo:38:kworker.0 00000100:00100000:2.0:1782488632.131370:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488632.131393:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300167040 00000100:00100000:0.0:1782488632.131401:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300167040 00000100:00100000:0.0:1782488632.131452:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 667 00000100:00100000:0.0:1782488632.131463:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455218850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300167040:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488632.131486:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300167168 00000100:00100000:2.0:1782488632.131492:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300167168 00000100:00100000:2.0:1782488632.131520:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300167296 00000100:00100000:0.0:1782488632.131521:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455218850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300167040:12345-0@lo:400:kworker.0 Request processed in 58us (253us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488632.131524:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300167296 00000100:00100000:0.0:1782488632.131531:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 667 00000100:00100000:2.0:1782488632.131547:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14172 00000100:00100000:2.0:1782488632.131554:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4bb704a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300167168:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488632.131571:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488632.131603:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb44ed09500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300167040:0@lo:400:kworker.0 00000100:00100000:2.0:1782488632.131626:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4bb704a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300167168:12345-0@lo:38:kworker.0 Request processed in 51us (284us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488632.131635:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14172 00000100:00100000:2.0:1782488632.131651:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14173 00000100:00100000:2.0:1782488632.131655:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4bb706a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300167296:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488632.131662:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488632.131686:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4bb706a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300167296:12345-0@lo:38:kworker.0 Request processed in 30us (316us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488632.131692:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14173 00000100:00080000:2.0:1782488632.131739:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488632.131743:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488632.131747:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb44ed09f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300167296:0@lo:38:kworker.0 00000100:00080000:2.0:1782488632.131800:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488632.131804:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488632.131807:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb44ed0ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300167168:0@lo:38:kworker.0 00010000:00010000:0.0:1782488632.961737:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488633.985743:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488635.009771:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488636.033747:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488637.057732:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488637.249753:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488637.249761:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488637.249766:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488637.249775:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8259 00000100:00080000:2.0:1782488637.249779:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488637.249787:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488637.249811:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4bb706d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300167424:0@lo:400:kworker.0 00000100:00100000:2.0:1782488637.249820:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb4bb704000 x1869066300167552/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488637.249850:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488637.249858:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488637.249860:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488637.249865:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8259 00000100:00080000:2.0:1782488637.249868:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488637.249873:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488637.249889:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb4bb706300 x1869066300167680/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488637.249917:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4bb704000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300167552:0@lo:38:kworker.0 00000100:00100000:0.0:1782488637.251055:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488637.251087:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488637.251125:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4bb706300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300167680:0@lo:38:kworker.0 00000100:00100000:0.0:1782488637.251157:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300167424 00000100:00100000:0.0:1782488637.251165:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300167424 00000100:00100000:2.0:1782488637.251177:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488637.251212:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 668 00000100:00100000:0.0:1782488637.251221:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45521ac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300167424:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488637.251253:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300167552 00000100:00100000:2.0:1782488637.251259:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300167552 00000100:00100000:0.0:1782488637.251279:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45521ac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300167424:12345-0@lo:400:kworker.0 Request processed in 58us (226us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488637.251284:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300167680 00000100:00100000:2.0:1782488637.251289:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300167680 00000100:00100000:0.0:1782488637.251290:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 668 00000100:00100000:2.0:1782488637.251315:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14174 00000100:00100000:2.0:1782488637.251323:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4bb706680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300167552:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488637.251337:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488637.251380:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4bb706680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300167552:12345-0@lo:38:kworker.0 Request processed in 57us (294us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488637.251387:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14174 00000100:00100000:2.0:1782488637.251406:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14175 00000100:00100000:2.0:1782488637.251413:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb545d91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300167680:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488637.251421:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488637.251466:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb545d91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300167680:12345-0@lo:38:kworker.0 Request processed in 53us (290us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488637.251473:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14175 00000100:00080000:2.0:1782488637.251528:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488637.251534:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488637.251539:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4bb704000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300167552:0@lo:38:kworker.0 00000100:00100000:0.0:1782488637.251604:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4bb706d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300167424:0@lo:400:kworker.0 00000100:00080000:2.0:1782488637.251770:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488637.251776:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488637.251780:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4bb706300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300167680:0@lo:38:kworker.0 00010000:00010000:0.0:1782488638.081718:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488639.105727:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488640.129732:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488641.153757:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488642.177725:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488642.369778:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488642.369785:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488642.369790:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1782488642.369792:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb44ed09500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300167808:0@lo:400:kworker.0 00000100:00080000:0.0:1782488642.369798:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8264 00000100:00080000:0.0:1782488642.369804:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488642.369812:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488642.369837:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb44ed0bb80 x1869066300167936/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1782488642.369888:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1782488642.369893:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1782488642.369894:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488642.369899:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488642.369904:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8264 00000100:00080000:0.0:1782488642.369908:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488642.369912:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488642.369935:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb44ed0a300 x1869066300168064/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488642.369983:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300167808 00000100:00100000:2.0:1782488642.369990:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300167808 00000100:00100000:0.0:1782488642.369998:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb44ed0bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300167936:0@lo:38:kworker.0 00000100:00100000:0.0:1782488642.370045:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488642.370055:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 669 00000100:00100000:0.0:1782488642.370069:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb44ed0a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300168064:0@lo:38:kworker.0 00000100:00100000:2.0:1782488642.370085:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455219850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300167808:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488642.370093:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488642.370157:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455219850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300167808:12345-0@lo:400:kworker.0 Request processed in 74us (267us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488642.370164:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 669 00000100:00100000:0.0:1782488642.370179:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300167936 00000100:00100000:0.0:1782488642.370183:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300167936 00000100:00100000:2.0:1782488642.370211:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb44ed09500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300167808:0@lo:400:kworker.0 00000100:00100000:0.0:1782488642.370220:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300168064 00000100:00100000:0.0:1782488642.370224:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300168064 00000100:00100000:0.0:1782488642.370246:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14176 00000100:00100000:0.0:1782488642.370253:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb44ed09180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300167936:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488642.370268:0:190296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488642.370307:0:213935:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14177 00000100:00100000:0.0:1782488642.370314:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb44ed09180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300167936:12345-0@lo:38:kworker.0 Request processed in 58us (266us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488642.370314:0:213935:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb44ed0ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300168064:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488642.370328:0:213935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488642.370331:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14176 00000100:00100000:2.0:1782488642.370358:0:213935:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb44ed0ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300168064:12345-0@lo:38:kworker.0 Request processed in 43us (265us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488642.370363:0:213935:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14177 00000100:00080000:0.0:1782488642.370673:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488642.370680:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488642.370685:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb44ed0bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300167936:0@lo:38:kworker.0 00000100:00080000:0.0:1782488642.370709:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488642.370714:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488642.370718:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb44ed0a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300168064:0@lo:38:kworker.0 00010000:00010000:0.0:1782488643.201756:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488644.225728:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488645.249756:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488646.273749:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488647.297719:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488647.489760:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488647.489768:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00100000:0.0:1782488647.489772:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb545d90a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300168192:0@lo:400:kworker.0 00000100:00080000:2.0:1782488647.489774:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488647.489783:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8269 00000100:00080000:2.0:1782488647.489788:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488647.489796:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488647.489818:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb545d92a00 x1869066300168320/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488647.489839:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488647.489843:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488647.489845:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488647.489852:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8269 00000100:00100000:0.0:1782488647.489857:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1782488647.489857:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488647.489864:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488647.489931:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb452cc8a80 x1869066300168448/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488647.490006:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300168192 00000100:00100000:2.0:1782488647.490013:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300168192 00000100:00100000:2.0:1782488647.490062:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 670 00000100:00100000:2.0:1782488647.490073:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45521b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300168192:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488647.490137:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45521b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300168192:12345-0@lo:400:kworker.0 Request processed in 65us (284us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488647.490145:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 670 00000100:00100000:2.0:1782488647.490186:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb545d92a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300168320:0@lo:38:kworker.0 00000100:00100000:2.0:1782488647.490220:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488647.490244:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb545d90a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300168192:0@lo:400:kworker.0 00000100:00100000:2.0:1782488647.490249:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb452cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300168448:0@lo:38:kworker.0 00000100:00100000:2.0:1782488647.490274:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488647.490308:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300168320 00000100:00100000:2.0:1782488647.490310:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300168320 00000100:00100000:2.0:1782488647.490331:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300168448 00000100:00100000:2.0:1782488647.490333:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300168448 00000100:00100000:2.0:1782488647.490354:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14178 00000100:00100000:2.0:1782488647.490360:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb452cc8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300168320:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488647.490374:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488647.490417:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb452cc8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300168320:12345-0@lo:38:kworker.0 Request processed in 55us (197us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488647.490423:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14178 00000100:00100000:2.0:1782488647.490438:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14179 00000100:00100000:2.0:1782488647.490442:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb452ccb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300168448:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488647.490456:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488647.490490:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb452ccb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300168448:12345-0@lo:38:kworker.0 Request processed in 47us (217us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488647.490497:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14179 00000100:00080000:2.0:1782488647.490534:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488647.490538:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488647.490541:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb545d92a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300168320:0@lo:38:kworker.0 00000100:00080000:2.0:1782488647.490551:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488647.490554:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488647.490558:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb452cc8a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300168448:0@lo:38:kworker.0 00010000:00010000:1.0:1782488648.321734:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488649.345889:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488650.369737:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1782488651.393758:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488652.417716:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1782488652.609760:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488652.609766:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488652.609769:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488652.609775:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8274 00000100:00080000:1.0:1782488652.609779:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488652.609785:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488652.609804:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb4cf55ad80 x1869066300168704/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:1.0:1782488652.609819:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1782488652.609822:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1782488652.609824:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1782488652.609827:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8274 00000100:00080000:1.0:1782488652.609829:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1782488652.609832:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:1.0:1782488652.609855:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb4cf55a300 x1869066300168832/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:1.0:1782488652.609880:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4cf558380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300168576:0@lo:400:kworker.0 00000100:00100000:2.0:1782488652.609988:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4cf55ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300168704:0@lo:38:kworker.0 00000100:00100000:2.0:1782488652.610042:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488652.610099:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4cf55a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300168832:0@lo:38:kworker.0 00000100:00100000:1.0:1782488652.610165:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488652.610178:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488652.610236:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300168704 00000100:00100000:2.0:1782488652.610248:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300168704 00000100:00100000:2.0:1782488652.610287:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300168832 00000100:00100000:2.0:1782488652.610291:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300168832 00000100:00100000:1.0:1782488652.610317:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300168576 00000100:00100000:1.0:1782488652.610323:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300168576 00000100:00100000:2.0:1782488652.610328:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14180 00000100:00100000:2.0:1782488652.610337:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb452cc8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300168704:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488652.610354:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1782488652.610372:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 671 00000100:00100000:1.0:1782488652.610382:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45521b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300168576:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488652.610403:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb452cc8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300168704:12345-0@lo:38:kworker.0 Request processed in 63us (359us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488652.610412:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14180 00000100:00100000:1.0:1782488652.610432:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45521b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300168576:12345-0@lo:400:kworker.0 Request processed in 50us (266us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488652.610433:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14181 00000100:00100000:2.0:1782488652.610443:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4a41d9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300168832:12345-0@lo:38:kworker.0 00000100:00100000:1.0:1782488652.610446:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 671 00010000:02000000:2.0:1782488652.610474:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488652.610513:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4a41d9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300168832:12345-0@lo:38:kworker.0 Request processed in 68us (333us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488652.610521:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14181 00000100:00100000:1.0:1782488652.610534:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4cf558380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300168576:0@lo:400:kworker.0 00000100:00080000:2.0:1782488652.610564:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488652.610569:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488652.610572:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4cf55ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300168704:0@lo:38:kworker.0 00000100:00080000:2.0:1782488652.610584:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488652.610587:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488652.610590:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4cf55a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300168832:0@lo:38:kworker.0 00010000:00010000:0.0:1782488653.441749:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488654.465709:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488655.489722:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488656.513720:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488657.537728:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488657.729865:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488657.729872:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488657.729877:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488657.729886:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8280 00000100:00080000:0.0:1782488657.729902:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488657.729909:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488657.729934:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb44ed0bb80 x1869066300169088/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1782488657.729950:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488657.729952:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488657.729954:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488657.729994:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8280 00000100:00080000:0.0:1782488657.729997:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488657.730000:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488657.730033:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb44ed0a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300168960:0@lo:400:kworker.0 00000100:00100000:2.0:1782488657.730105:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488657.730201:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb44ed0bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300169088:0@lo:38:kworker.0 00000100:00100000:2.0:1782488657.730252:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488657.730304:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300169088 00000100:00100000:0.0:1782488657.730308:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300168960 00000100:00100000:2.0:1782488657.730311:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300169088 00000100:00100000:0.0:1782488657.730313:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300168960 00000100:00100000:2.0:1782488657.730352:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14182 00000100:00100000:2.0:1782488657.730361:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4a41daa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300169088:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488657.730379:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488657.730385:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 672 00000100:00100000:0.0:1782488657.730394:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45521bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300168960:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488657.730445:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4a41daa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300169088:12345-0@lo:38:kworker.0 Request processed in 82us (191us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488657.730453:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45521bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300168960:12345-0@lo:400:kworker.0 Request processed in 57us (346us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488657.730460:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14182 00000100:00100000:0.0:1782488657.730465:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 672 00000100:00080000:2.0:1782488657.730513:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488657.730520:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488657.730551:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb44ed0bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300169088:0@lo:38:kworker.0 00000100:00100000:0.0:1782488657.730587:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb44ed0a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300168960:0@lo:400:kworker.0 00000100:00100000:0.0:1782488657.730643:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb44ed0a300 x1869066300169216/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488657.730676:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb44ed0a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300169216:0@lo:38:kworker.0 00000100:00100000:0.0:1782488657.730706:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488657.730766:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300169216 00000100:00100000:2.0:1782488657.730772:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300169216 00000100:00100000:2.0:1782488657.730801:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14183 00000100:00100000:2.0:1782488657.730807:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb44ed09f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300169216:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488657.730816:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488657.730859:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb44ed09f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300169216:12345-0@lo:38:kworker.0 Request processed in 50us (153us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488657.730868:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14183 00000100:00080000:2.0:1782488657.730917:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1782488657.730923:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488657.730926:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb44ed0a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300169216:0@lo:38:kworker.0 00010000:00010000:3.0:1782488658.561706:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488659.585714:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1782488660.609701:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488661.633725:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488662.657779:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488662.849777:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488662.849782:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488662.849786:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488662.849792:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8285 00000100:00080000:2.0:1782488662.849795:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488662.849801:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488662.849819:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb4a41dbb80 x1869066300169472/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488662.849834:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488662.849837:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488662.849839:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488662.849842:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8285 00000100:00080000:2.0:1782488662.849865:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488662.849869:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488662.849883:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb4a41d8700 x1869066300169600/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488662.849913:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4a41d9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300169344:0@lo:400:kworker.0 00000100:00100000:2.0:1782488662.849986:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488662.850097:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4a41dbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300169472:0@lo:38:kworker.0 00000100:00100000:0.0:1782488662.850144:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488662.850188:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4a41d8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300169600:0@lo:38:kworker.0 00000100:00100000:2.0:1782488662.850197:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300169344 00000100:00100000:2.0:1782488662.850206:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300169344 00000100:00100000:0.0:1782488662.850219:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488662.850300:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300169472 00000100:00100000:0.0:1782488662.850305:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300169472 00000100:00100000:0.0:1782488662.850333:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300169600 00000100:00100000:0.0:1782488662.850337:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300169600 00000100:00100000:0.0:1782488662.850372:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14184 00000100:00100000:0.0:1782488662.850380:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb44ed08380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300169472:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488662.850397:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488662.850408:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 673 00000100:00100000:2.0:1782488662.850417:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45521a050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300169344:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488662.850460:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45521a050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300169344:12345-0@lo:400:kworker.0 Request processed in 43us (474us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488662.850463:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb44ed08380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300169472:12345-0@lo:38:kworker.0 Request processed in 77us (313us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488662.850470:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 673 00000100:00100000:0.0:1782488662.850476:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14184 00000100:00100000:0.0:1782488662.850496:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14185 00000100:00100000:0.0:1782488662.850502:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb44ed0a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300169600:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488662.850511:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488662.850548:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb44ed0a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300169600:12345-0@lo:38:kworker.0 Request processed in 43us (328us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488662.850550:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4a41d9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300169344:0@lo:400:kworker.0 00000100:00100000:0.0:1782488662.850561:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14185 00000100:00080000:0.0:1782488662.850632:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488662.850639:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488662.850643:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4a41dbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300169472:0@lo:38:kworker.0 00000100:00080000:0.0:1782488662.850658:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488662.850663:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488662.850667:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4a41d8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300169600:0@lo:38:kworker.0 00010000:00010000:0.0:1782488663.682704:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1782488664.705714:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488665.729727:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488666.753769:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488667.777723:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488667.969824:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488667.969832:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488667.969836:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488667.969844:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8290 00000100:00080000:2.0:1782488667.969849:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488667.969858:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488667.969891:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb4a41d9500 x1869066300169856/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488667.969912:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488667.969917:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488667.969932:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488667.969938:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8290 00000100:00080000:2.0:1782488667.969941:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488667.969947:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488667.969963:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb4a41d8e00 x1869066300169984/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488667.969996:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4a41d9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300169728:0@lo:400:kworker.0 00000100:00100000:2.0:1782488667.970069:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488667.970151:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4a41d9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300169856:0@lo:38:kworker.0 00000100:00100000:2.0:1782488667.970187:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488667.970225:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4a41d8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300169984:0@lo:38:kworker.0 00000100:00100000:2.0:1782488667.970255:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488667.970306:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300169728 00000100:00100000:2.0:1782488667.970313:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300169728 00000100:00100000:2.0:1782488667.970371:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 674 00000100:00100000:2.0:1782488667.970383:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455219450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300169728:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488667.970452:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455219450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300169728:12345-0@lo:400:kworker.0 Request processed in 70us (383us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488667.970454:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300169856 00000100:00100000:0.0:1782488667.970462:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300169856 00000100:00100000:2.0:1782488667.970466:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 674 00000100:00100000:0.0:1782488667.970494:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300169984 00000100:00100000:0.0:1782488667.970498:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300169984 00000100:00100000:0.0:1782488667.970526:0:213935:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14186 00000100:00100000:2.0:1782488667.970533:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4a41d9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300169728:0@lo:400:kworker.0 00000100:00100000:0.0:1782488667.970534:0:213935:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4a41d8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300169856:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488667.970555:0:213935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488667.970585:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14187 00000100:00100000:2.0:1782488667.970593:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4a41d8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300169984:12345-0@lo:38:kworker.0 00000100:00100000:0.0:1782488667.970601:0:213935:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4a41d8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300169856:12345-0@lo:38:kworker.0 Request processed in 66us (413us total) trans 0 rc -19/-19 00010000:02000000:2.0:1782488667.970604:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488667.970640:0:213935:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14186 00000100:00100000:2.0:1782488667.970670:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4a41d8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300169984:12345-0@lo:38:kworker.0 Request processed in 75us (414us total) trans 0 rc -19/-19 00000100:00080000:0.0:1782488667.970686:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:2.0:1782488667.970690:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14187 00000100:00080000:0.0:1782488667.970695:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488667.970700:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4a41d9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300169856:0@lo:38:kworker.0 00000100:00080000:0.0:1782488667.970717:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488667.970722:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488667.970727:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4a41d8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300169984:0@lo:38:kworker.0 00010000:00010000:0.0:1782488668.801741:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488669.825739:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488670.849792:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488671.873809:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488672.897813:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488673.089918:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488673.089925:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488673.089929:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488673.089937:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8295 00000100:00080000:0.0:1782488673.089942:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488673.089949:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488673.089969:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb44ed0a300 x1869066300170240/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1782488673.089988:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488673.089992:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488673.090006:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488673.090011:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8295 00000100:00080000:0.0:1782488673.090014:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488673.090018:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488673.090032:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb44ed0bb80 x1869066300170368/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488673.090110:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb44ed08000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300170112:0@lo:400:kworker.0 00000100:00100000:0.0:1782488673.090120:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb44ed0a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300170240:0@lo:38:kworker.0 00000100:00100000:0.0:1782488673.090194:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488673.090235:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488673.090244:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb44ed0bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300170368:0@lo:38:kworker.0 00000100:00100000:0.0:1782488673.090283:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488673.090313:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300170112 00000100:00100000:2.0:1782488673.090321:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300170112 00000100:00100000:0.0:1782488673.090348:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300170240 00000100:00100000:0.0:1782488673.090352:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300170240 00000100:00100000:2.0:1782488673.090379:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 675 00000100:00100000:0.0:1782488673.090380:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300170368 00000100:00100000:0.0:1782488673.090382:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300170368 00000100:00100000:2.0:1782488673.090390:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455218c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300170112:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488673.090413:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14188 00000100:00100000:0.0:1782488673.090419:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb44ed09500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300170240:12345-0@lo:38:kworker.0 00000100:00100000:2.0:1782488673.090438:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455218c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300170112:12345-0@lo:400:kworker.0 Request processed in 50us (246us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488673.090451:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 675 00010000:02000000:0.0:1782488673.090454:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488673.090492:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb44ed09500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300170240:12345-0@lo:38:kworker.0 Request processed in 72us (299us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488673.090500:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14188 00000100:00100000:0.0:1782488673.090517:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14189 00000100:00100000:0.0:1782488673.090522:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb545d92a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300170368:12345-0@lo:38:kworker.0 00000100:00100000:2.0:1782488673.090524:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb44ed08000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300170112:0@lo:400:kworker.0 00010000:02000000:0.0:1782488673.090529:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488673.090559:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb545d92a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300170368:12345-0@lo:38:kworker.0 Request processed in 35us (276us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488673.090566:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14189 00000100:00080000:0.0:1782488673.090631:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488673.090655:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488673.090660:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb44ed0a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300170240:0@lo:38:kworker.0 00000100:00080000:0.0:1782488673.090676:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488673.090680:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488673.090685:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb44ed0bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300170368:0@lo:38:kworker.0 00010000:00010000:0.0:1782488673.921789:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488674.945719:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488675.969713:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488676.993740:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488678.017785:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488678.210166:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488678.210173:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488678.210178:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488678.210187:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8300 00000100:00080000:0.0:1782488678.210192:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488678.210199:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488678.210246:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb4bb706300 x1869066300170624/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1782488678.210280:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488678.210285:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488678.210287:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488678.210291:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8300 00000100:00080000:0.0:1782488678.210294:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488678.210300:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488678.210315:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb4bb704000 x1869066300170752/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488678.210358:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb545d90a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300170496:0@lo:400:kworker.0 00000100:00100000:0.0:1782488678.210466:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488678.210540:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4bb706300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300170624:0@lo:38:kworker.0 00000100:00100000:0.0:1782488678.210573:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488678.210599:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4bb704000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300170752:0@lo:38:kworker.0 00000100:00100000:0.0:1782488678.210650:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488678.210724:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300170496 00000100:00100000:0.0:1782488678.210731:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300170496 00000100:00100000:0.0:1782488678.210785:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 676 00000100:00100000:0.0:1782488678.210795:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb455218450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300170496:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488678.210844:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300170624 00000100:00100000:0.0:1782488678.210847:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb455218450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300170496:12345-0@lo:400:kworker.0 Request processed in 52us (381us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488678.210855:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300170624 00000100:00100000:0.0:1782488678.210859:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 676 00000100:00100000:2.0:1782488678.210906:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300170752 00000100:00100000:2.0:1782488678.210910:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300170752 00000100:00100000:0.0:1782488678.210924:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14190 00000100:00100000:0.0:1782488678.210933:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4bb706d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300170624:12345-0@lo:38:kworker.0 00000100:00100000:2.0:1782488678.210937:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14191 00000100:00100000:2.0:1782488678.210944:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb451b2aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300170752:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488678.210951:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00010000:02000000:2.0:1782488678.210954:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488678.210996:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4bb706d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300170624:12345-0@lo:38:kworker.0 Request processed in 62us (422us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488678.211003:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb451b2aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300170752:12345-0@lo:38:kworker.0 Request processed in 56us (351us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488678.211008:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14190 00000100:00100000:2.0:1782488678.211013:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14191 00000100:00080000:0.0:1782488678.211090:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488678.211097:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488678.211102:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4bb706300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300170624:0@lo:38:kworker.0 00000100:00080000:0.0:1782488678.211113:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488678.211118:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488678.211129:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4bb704000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300170752:0@lo:38:kworker.0 00000100:00100000:0.0:1782488678.211169:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb545d90a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300170496:0@lo:400:kworker.0 00010000:00010000:0.0:1782488679.041735:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488680.065780:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488681.089760:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488682.113824:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488683.137746:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488683.329820:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488683.329827:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488683.329830:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488683.329838:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8305 00000100:00080000:0.0:1782488683.329843:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 5s 00000100:00100000:2.0:1782488683.329849:0:3315:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00080000:0.0:1782488683.329850:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488683.329872:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb451b28700 x1869066300171008/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1782488683.329887:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488683.329890:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488683.329894:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1782488683.329894:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb451b2b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300170880:0@lo:400:kworker.0 00000100:00080000:0.0:1782488683.329901:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8305 00000100:00080000:0.0:1782488683.329905:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488683.329908:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488683.329923:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb545d90a80 x1869066300171136/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488683.329955:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb451b28700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300171008:0@lo:38:kworker.0 00000100:00100000:2.0:1782488683.329982:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488683.330010:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488683.330067:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb545d90a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300171136:0@lo:38:kworker.0 00000100:00100000:2.0:1782488683.330070:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300170880 00000100:00100000:2.0:1782488683.330079:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300170880 00000100:00100000:0.0:1782488683.330092:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488683.330138:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 677 00000100:00100000:2.0:1782488683.330150:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb45521a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300170880:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488683.330156:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300171008 00000100:00100000:0.0:1782488683.330161:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300171008 00000100:00100000:0.0:1782488683.330193:0:213935:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300171136 00000100:00100000:0.0:1782488683.330196:0:213935:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300171136 00000100:00100000:2.0:1782488683.330208:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb45521a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300170880:12345-0@lo:400:kworker.0 Request processed in 61us (227us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488683.330217:0:213935:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14192 00000100:00100000:2.0:1782488683.330220:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 677 00000100:00100000:0.0:1782488683.330226:0:213935:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4bb704000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300171008:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488683.330241:0:213935:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488683.330277:0:213935:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4bb704000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_007:0+-99:3313:x1869066300171008:12345-0@lo:38:kworker.0 Request processed in 51us (267us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488683.330284:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14193 00000100:00100000:0.0:1782488683.330291:0:213935:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14192 00000100:00100000:2.0:1782488683.330295:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4bb706300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300171136:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488683.330306:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00080000:0.0:1782488683.330334:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488683.330339:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:2.0:1782488683.330344:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4bb706300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300171136:12345-0@lo:38:kworker.0 Request processed in 49us (249us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488683.330346:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb451b28700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300171008:0@lo:38:kworker.0 00000100:00100000:2.0:1782488683.330359:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14193 00000100:00080000:0.0:1782488683.330363:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488683.330369:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488683.330372:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb545d90a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300171136:0@lo:38:kworker.0 00000100:00100000:0.0:1782488683.330693:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb451b2b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300170880:0@lo:400:kworker.0 00010000:00010000:0.0:1782488684.161756:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488685.185778:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488686.209744:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488687.233766:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488688.257739:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488688.449818:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488688.449823:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488688.449826:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488688.449834:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8310 00000100:00080000:2.0:1782488688.449839:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 10s 00000100:00080000:2.0:1782488688.449845:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488688.449870:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb4a41d8380 x1869066300171392/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488688.449887:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488688.449890:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488688.449892:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488688.449896:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8310 00000100:00080000:2.0:1782488688.449898:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488688.449902:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488688.449939:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb4a41db480 x1869066300171520/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488688.449971:0:3314:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1782488688.449998:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4a41d9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300171264:0@lo:400:kworker.0 00000100:00100000:2.0:1782488688.450074:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488688.450102:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4a41d8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300171392:0@lo:38:kworker.0 00000100:00100000:0.0:1782488688.450144:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488688.450158:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300171264 00000100:00100000:2.0:1782488688.450166:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300171264 00000100:00100000:0.0:1782488688.450187:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4a41db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300171520:0@lo:38:kworker.0 00000100:00100000:0.0:1782488688.450212:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488688.450216:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 678 00000100:00100000:2.0:1782488688.450236:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579add850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300171264:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488688.450284:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300171392 00000100:00100000:0.0:1782488688.450289:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300171392 00000100:00100000:0.0:1782488688.450327:0:190298:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300171520 00000100:00100000:0.0:1782488688.450332:0:190298:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300171520 00000100:00100000:2.0:1782488688.450347:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579add850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300171264:12345-0@lo:400:kworker.0 Request processed in 121us (279us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488688.450355:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 678 00000100:00100000:0.0:1782488688.450365:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14194 00000100:00100000:0.0:1782488688.450374:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb545d90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300171392:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488688.450390:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488688.450405:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4a41d9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300171264:0@lo:400:kworker.0 00000100:00100000:0.0:1782488688.450433:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb545d90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300171392:12345-0@lo:38:kworker.0 Request processed in 58us (287us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488688.450439:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14194 00000100:00100000:0.0:1782488688.450457:0:190298:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14195 00000100:00100000:0.0:1782488688.450463:0:190298:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb451b2b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300171520:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488688.450471:0:190298:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488688.450503:0:190298:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb451b2b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3313:x1869066300171520:12345-0@lo:38:kworker.0 Request processed in 38us (291us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488688.450511:0:190298:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14195 00000100:00080000:0.0:1782488688.450544:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488688.450549:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488688.450552:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4a41d8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300171392:0@lo:38:kworker.0 00000100:00080000:0.0:1782488688.450561:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488688.450564:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488688.450568:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4a41db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300171520:0@lo:38:kworker.0 00010000:00010000:0.0:1782488689.281790:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488690.305776:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488691.329734:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488692.353729:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488693.377745:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488693.569747:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488693.569752:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488693.569756:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488693.569763:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8315 00000100:00080000:0.0:1782488693.569767:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 15s 00000100:00080000:0.0:1782488693.569774:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488693.569795:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb57ff2ea00 x1869066300171776/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1782488693.569813:0:186303:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488693.569817:0:186303:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488693.569820:0:186303:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488693.569824:0:186303:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8315 00000100:00080000:0.0:1782488693.569827:0:186303:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 5s 00000100:00080000:0.0:1782488693.569832:0:186303:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488693.569848:0:186303:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb57ff2e300 x1869066300171904/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488693.569882:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb451b28700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300171648:0@lo:400:kworker.0 00000100:00100000:0.0:1782488693.569948:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488693.570030:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb57ff2ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300171776:0@lo:38:kworker.0 00000100:00100000:0.0:1782488693.570063:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488693.570088:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb57ff2e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300171904:0@lo:38:kworker.0 00000100:00100000:0.0:1782488693.570132:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488693.570196:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300171648 00000100:00100000:0.0:1782488693.570224:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300171648 00000100:00100000:0.0:1782488693.570276:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 679 00000100:00100000:0.0:1782488693.570287:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579adc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300171648:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488693.570341:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579adc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300171648:12345-0@lo:400:kworker.0 Request processed in 56us (394us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488693.570357:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 679 00000100:00100000:0.0:1782488693.570403:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300171776 00000100:00100000:0.0:1782488693.570407:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300171776 00000100:00100000:0.0:1782488693.570431:0:213684:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300171904 00000100:00100000:0.0:1782488693.570435:0:213684:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300171904 00000100:00100000:0.0:1782488693.570457:0:213684:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14196 00000100:00100000:0.0:1782488693.570464:0:213684:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57ff2c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300171776:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488693.570479:0:213684:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488693.570497:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14197 00000100:00100000:2.0:1782488693.570506:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57ff2dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300171904:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488693.570517:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488693.570567:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57ff2dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300171904:12345-0@lo:38:kworker.0 Request processed in 61us (435us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488693.570575:0:213684:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57ff2c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:0+-99:3313:x1869066300171776:12345-0@lo:38:kworker.0 Request processed in 108us (510us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488693.570590:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14197 00000100:00100000:0.0:1782488693.570601:0:213684:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14196 00000100:00080000:0.0:1782488693.570662:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488693.570668:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488693.570673:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb57ff2ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300171776:0@lo:38:kworker.0 00000100:00080000:0.0:1782488693.570687:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488693.570692:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488693.570696:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb57ff2e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300171904:0@lo:38:kworker.0 00000100:00100000:0.0:1782488693.570730:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb451b28700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300171648:0@lo:400:kworker.0 00010000:00010000:0.0:1782488694.401821:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488695.425769:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488696.449742:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488697.473754:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488698.497736:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488698.689753:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488698.689759:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488698.689762:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488698.689769:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8320 00000100:00080000:2.0:1782488698.689773:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488698.689779:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488698.689818:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb4a41dad80 x1869066300172160/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488698.689836:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488698.689839:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488698.689841:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488698.689845:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8320 00000100:00080000:2.0:1782488698.689848:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 10s 00000100:00080000:2.0:1782488698.689852:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488698.689865:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb4a41db480 x1869066300172288/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488698.689904:0:3316:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1782488698.689922:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4a41d9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300172032:0@lo:400:kworker.0 00000100:00100000:0.0:1782488698.689993:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4a41dad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300172160:0@lo:38:kworker.0 00000100:00100000:0.0:1782488698.690046:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488698.690104:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4a41db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300172288:0@lo:38:kworker.0 00000100:00100000:0.0:1782488698.690136:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488698.690155:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488698.690184:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300172160 00000100:00100000:0.0:1782488698.690191:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300172160 00000100:00100000:0.0:1782488698.690219:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300172288 00000100:00100000:0.0:1782488698.690222:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300172288 00000100:00100000:0.0:1782488698.690248:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14198 00000100:00100000:0.0:1782488698.690257:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57ff2e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300172160:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488698.690274:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488698.690322:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300172032 00000100:00100000:2.0:1782488698.690327:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300172032 00000100:00100000:0.0:1782488698.690328:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57ff2e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300172160:12345-0@lo:38:kworker.0 Request processed in 65us (276us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488698.690339:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14198 00000100:00100000:0.0:1782488698.690363:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14199 00000100:00100000:0.0:1782488698.690372:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57ff2ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300172288:12345-0@lo:38:kworker.0 00000100:00100000:2.0:1782488698.690372:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 680 00010000:02000000:0.0:1782488698.690382:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488698.690387:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579adf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300172032:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488698.690414:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57ff2ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300172288:12345-0@lo:38:kworker.0 Request processed in 42us (276us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488698.690420:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14199 00000100:00100000:2.0:1782488698.690433:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579adf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300172032:12345-0@lo:400:kworker.0 Request processed in 50us (279us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488698.690439:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 680 00000100:00080000:0.0:1782488698.690485:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:2.0:1782488698.690489:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4a41d9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300172032:0@lo:400:kworker.0 00000100:00080000:0.0:1782488698.690495:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488698.690502:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4a41dad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300172160:0@lo:38:kworker.0 00000100:00080000:0.0:1782488698.690528:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488698.690543:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488698.690583:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4a41db480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300172288:0@lo:38:kworker.0 00010000:00010000:0.0:1782488699.521736:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488700.547368:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488701.569748:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488702.593730:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488703.617751:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488703.810337:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488703.810345:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488703.810349:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488703.810357:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8326 00000100:00080000:0.0:1782488703.810362:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488703.810368:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488703.810393:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb57ff2df80 x1869066300172544/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1782488703.810412:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488703.810416:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488703.810418:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488703.810422:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8326 00000100:00080000:0.0:1782488703.810426:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 15s 00000100:00080000:0.0:1782488703.810430:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488703.810445:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb57ff2fb80 x1869066300172672/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488703.810475:0:3314:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb57ff2c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300172416:0@lo:400:kworker.0 00000100:00100000:0.0:1782488703.810561:0:3314:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488703.810719:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300172416 00000100:00100000:2.0:1782488703.810728:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300172416 00000100:00100000:0.0:1782488703.810735:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb57ff2df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300172544:0@lo:38:kworker.0 00000100:00100000:2.0:1782488703.810781:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 681 00000100:00100000:2.0:1782488703.810796:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579adf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300172416:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488703.810819:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488703.810847:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb57ff2fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300172672:0@lo:38:kworker.0 00000100:00100000:0.0:1782488703.810872:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488703.810930:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300172544 00000100:00100000:0.0:1782488703.810935:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300172544 00000100:00100000:0.0:1782488703.810961:0:196437:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300172672 00000100:00100000:2.0:1782488703.810963:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579adf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3314:x1869066300172416:12345-0@lo:400:kworker.0 Request processed in 168us (400us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488703.810966:0:196437:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300172672 00000100:00100000:2.0:1782488703.810975:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 681 00000100:00100000:0.0:1782488703.811002:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14200 00000100:00100000:0.0:1782488703.811009:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57ff2e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300172544:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488703.811024:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488703.811083:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57ff2e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300172544:12345-0@lo:38:kworker.0 Request processed in 71us (263us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488703.811090:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14200 00000100:00100000:0.0:1782488703.811107:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14201 00000100:00100000:0.0:1782488703.811111:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57ff2f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300172672:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488703.811119:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488703.811149:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57ff2f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300172672:12345-0@lo:38:kworker.0 Request processed in 35us (276us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488703.811155:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14201 00000100:00100000:0.0:1782488703.811217:0:3314:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb57ff2c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:6385baa4-edbe-4eae-af92-80250729af95:3314:1869066300172416:0@lo:400:kworker.0 00000100:00080000:0.0:1782488703.811261:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488703.811267:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488703.811270:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb57ff2df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300172544:0@lo:38:kworker.0 00000100:00080000:0.0:1782488703.811281:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488703.811285:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488703.811289:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb57ff2fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300172672:0@lo:38:kworker.0 00010000:00010000:0.0:1782488704.641741:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488705.666709:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488706.689722:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488707.713779:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488708.737734:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488708.929794:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488708.929801:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488708.929806:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488708.929821:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8331 00000100:00080000:0.0:1782488708.929826:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488708.929833:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488708.929855:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb57ff2df80 x1869066300172928/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1782488708.929875:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488708.929879:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488708.929882:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488708.929887:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8331 00000100:00080000:0.0:1782488708.929890:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488708.929902:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488708.929918:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb57ff2c000 x1869066300173056/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488708.929971:0:3315:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb57ff2fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300172800:0@lo:400:kworker.0 00000100:00100000:0.0:1782488708.930050:0:3315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488708.930145:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb57ff2df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300172928:0@lo:38:kworker.0 00000100:00100000:0.0:1782488708.930182:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488708.930221:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb57ff2c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300173056:0@lo:38:kworker.0 00000100:00100000:0.0:1782488708.930251:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488708.930299:0:190292:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300172800 00000100:00100000:0.0:1782488708.930306:0:190292:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300172800 00000100:00100000:0.0:1782488708.930357:0:190292:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 682 00000100:00100000:0.0:1782488708.930366:0:190292:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579adf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300172800:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1782488708.930399:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300172928 00000100:00100000:2.0:1782488708.930404:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300172928 00000100:00100000:0.0:1782488708.930421:0:190292:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579adf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:6385baa4-edbe-4eae-af92-80250729af95+14:3315:x1869066300172800:12345-0@lo:400:kworker.0 Request processed in 54us (371us total) trans 0 rc 0/0 00000100:00100000:0.0:1782488708.930428:0:190292:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 682 00000100:00100000:2.0:1782488708.930484:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300173056 00000100:00100000:2.0:1782488708.930489:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300173056 00000100:00100000:0.0:1782488708.930492:0:196437:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14202 00000100:00100000:0.0:1782488708.930500:0:196437:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57ff2d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300172928:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488708.930515:0:196437:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488708.930519:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14203 00000100:00100000:2.0:1782488708.930527:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57ff2d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300173056:12345-0@lo:38:kworker.0 00010000:02000000:2.0:1782488708.930538:0:190296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488708.930560:0:196437:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57ff2d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3313:x1869066300172928:12345-0@lo:38:kworker.0 Request processed in 58us (377us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488708.930568:0:196437:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14202 00000100:00100000:2.0:1782488708.930581:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57ff2d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300173056:12345-0@lo:38:kworker.0 Request processed in 53us (329us total) trans 0 rc -19/-19 00000100:00100000:2.0:1782488708.930590:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14203 00000100:00080000:0.0:1782488708.930640:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488708.930647:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488708.930653:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb57ff2df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300172928:0@lo:38:kworker.0 00000100:00080000:0.0:1782488708.930671:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488708.930676:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488708.930680:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb57ff2c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300173056:0@lo:38:kworker.0 00000100:00100000:0.0:1782488708.930720:0:3315:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb57ff2fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:6385baa4-edbe-4eae-af92-80250729af95:3315:1869066300172800:0@lo:400:kworker.0 00010000:00010000:0.0:1782488709.761711:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488710.785748:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488711.809728:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488712.833740:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488713.857731:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1782488714.049840:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488714.049848:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488714.049852:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488714.049861:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8336 00000100:00080000:0.0:1782488714.049866:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488714.049874:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488714.049900:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb57ff2c000 x1869066300173312/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:0.0:1782488714.049920:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1782488714.049924:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1782488714.049934:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1782488714.049939:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8336 00000100:00080000:0.0:1782488714.049942:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1782488714.049947:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:0.0:1782488714.049965:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb57ff2df80 x1869066300173440/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:0.0:1782488714.050003:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb57ff2c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300173312:0@lo:38:kworker.0 00000100:00100000:0.0:1782488714.050080:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488714.050088:0:3316:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb57ff2fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300173184:0@lo:400:kworker.0 00000100:00100000:2.0:1782488714.050132:0:3316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1782488714.050150:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb57ff2df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300173440:0@lo:38:kworker.0 00000100:00100000:2.0:1782488714.050178:0:190291:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300173184 00000100:00100000:2.0:1782488714.050186:0:190291:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300173184 00000100:00100000:0.0:1782488714.050221:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488714.050227:0:190291:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 683 00000100:00100000:2.0:1782488714.050235:0:190291:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579addc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300173184:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488714.050268:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300173312 00000100:00100000:0.0:1782488714.050273:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300173312 00000100:00100000:0.0:1782488714.050301:0:213218:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300173440 00000100:00100000:0.0:1782488714.050304:0:213218:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300173440 00000100:00100000:2.0:1782488714.050323:0:190291:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579addc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:6385baa4-edbe-4eae-af92-80250729af95+14:3316:x1869066300173184:12345-0@lo:400:kworker.0 Request processed in 88us (191us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488714.050331:0:190291:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 683 00000100:00100000:0.0:1782488714.050373:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14204 00000100:00100000:0.0:1782488714.050380:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57ff2d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300173312:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488714.050396:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488714.050432:0:3316:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb57ff2fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:6385baa4-edbe-4eae-af92-80250729af95:3316:1869066300173184:0@lo:400:kworker.0 00000100:00100000:0.0:1782488714.050463:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57ff2d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300173312:12345-0@lo:38:kworker.0 Request processed in 78us (381us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488714.050471:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14204 00000100:00100000:0.0:1782488714.050504:0:213218:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14205 00000100:00100000:0.0:1782488714.050510:0:213218:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb57ff2ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300173440:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488714.050518:0:213218:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1782488714.050553:0:213218:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb57ff2ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:0+-99:3313:x1869066300173440:12345-0@lo:38:kworker.0 Request processed in 41us (332us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488714.050560:0:213218:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14205 00000100:00080000:0.0:1782488714.050636:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488714.050641:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488714.050647:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb57ff2c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300173312:0@lo:38:kworker.0 00000100:00080000:0.0:1782488714.050661:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488714.050666:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488714.050669:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb57ff2df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300173440:0@lo:38:kworker.0 00010000:00010000:0.0:1782488714.883757:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488715.905710:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488716.929745:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488717.953721:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488718.977758:0:191970:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1782488719.169741:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488719.169748:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488719.169753:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488719.169807:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 8341 00000100:00080000:2.0:1782488719.169812:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488719.169820:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0000: import ffff9cb57f35c000 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488719.169844:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb4a41d8380 x1869066300173696/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00080000:2.0:1782488719.169872:0:223038:0:(pinger.c:231:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1782488719.169876:0:223038:0:(pinger.c:239:ptlrpc_pinger_process_import()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1782488719.169878:0:223038:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1782488719.169882:0:223038:0:(import.c:512:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 8341 00000100:00080000:2.0:1782488719.169885:0:223038:0:(import.c:591:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1782488719.169889:0:223038:0:(import.c:626:import_select_connection()) lustre-MDT0000-lwp-OST0001: import ffff9cb579ddc800 using connection 192.168.202.121@tcp/0@lo 00000100:00100000:2.0:1782488719.169901:0:223038:0:(import.c:828:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@ffff9cb4a41d8e00 x1869066300173824/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' uid:4294967295 gid:4294967295 projid:4294967295 00000100:00100000:2.0:1782488719.169930:0:3317:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4a41d9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300173568:0@lo:400:kworker.0 00000100:00100000:2.0:1782488719.170007:0:3317:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488719.170091:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4a41d8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300173696:0@lo:38:kworker.0 00000100:00100000:2.0:1782488719.170124:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488719.170148:0:3313:0:(client.c:1905:ptlrpc_send_new_req()) Sending RPC req@ffff9cb4a41d8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300173824:0@lo:38:kworker.0 00000100:00100000:2.0:1782488719.170177:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1782488719.170219:0:190290:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300173568 00000100:00100000:2.0:1782488719.170226:0:190290:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300173568 00000100:00100000:0.0:1782488719.170257:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300173696 00000100:00100000:0.0:1782488719.170264:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300173696 00000100:00100000:2.0:1782488719.170290:0:190290:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 684 00000100:00100000:0.0:1782488719.170294:0:190296:0:(service.c:2300:ptlrpc_server_handle_req_in()) unwrap req x1869066300173824 00000100:00100000:0.0:1782488719.170298:0:190296:0:(service.c:2365:ptlrpc_server_handle_req_in()) got req x1869066300173824 00000100:00100000:2.0:1782488719.170302:0:190290:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb579adfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300173568:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1782488719.170325:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14206 00000100:00100000:0.0:1782488719.170333:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4a41d9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300173696:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488719.170348:0:190296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488719.170388:0:190290:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb579adfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:6385baa4-edbe-4eae-af92-80250729af95+14:3317:x1869066300173568:12345-0@lo:400:kworker.0 Request processed in 88us (380us total) trans 0 rc 0/0 00000100:00100000:2.0:1782488719.170404:0:190290:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 684 00000100:00100000:0.0:1782488719.170408:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4a41d9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300173696:12345-0@lo:38:kworker.0 Request processed in 74us (283us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488719.170415:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14206 00000100:00100000:0.0:1782488719.170431:0:190296:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 14207 00000100:00100000:0.0:1782488719.170435:0:190296:0:(service.c:2551:ptlrpc_server_handle_request()) Handling RPC req@ffff9cb4a41d8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300173824:12345-0@lo:38:kworker.0 00010000:02000000:0.0:1782488719.170441:0:190296:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1782488719.170471:0:3317:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4a41d9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:6385baa4-edbe-4eae-af92-80250729af95:3317:1869066300173568:0@lo:400:kworker.0 00000100:00100000:0.0:1782488719.170486:0:190296:0:(service.c:2609:ptlrpc_server_handle_request()) Handled RPC req@ffff9cb4a41d8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3313:x1869066300173824:12345-0@lo:38:kworker.0 Request processed in 46us (306us total) trans 0 rc -19/-19 00000100:00100000:0.0:1782488719.170494:0:190296:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 14207 00000100:00080000:0.0:1782488719.170536:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb57f35c000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488719.170542:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488719.170546:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4a41d8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3313:1869066300173696:0@lo:38:kworker.0 00000100:00080000:0.0:1782488719.170561:0:3313:0:(import.c:65:import_set_state_nolock()) ffff9cb579ddc800 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1782488719.170566:0:3313:0:(import.c:1513:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 0@lo failed (-19) 00000100:00100000:0.0:1782488719.170570:0:3313:0:(client.c:2400:ptlrpc_check_set()) Completed RPC req@ffff9cb4a41d8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3313:1869066300173824:0@lo:38:kworker.0 00010000:00010000:0.0:1782488720.001731:0:213229:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1782488721.025747:0:190285:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:02000400:0.0:1782488721.217702:0:229334:0:(genops.c:1873:obd_exports_barrier()) lustre-MDT0000 is waiting for obd_unlinked_exports more than 256 seconds. The obd refcount = 5. Is it stuck? 00000020:02000400:0.0:1782488721.221995:0:229334:0:(genops.c:1840:print_export_data()) lustre-MDT0000: UNLINKED ffff9cb551430000 5c4ca15f-f34e-4111-b105-de67ffe3ec92 192.168.202.21@tcp 2 (0 0 0) 1 0 0 0: 0000000000000000 8589967277 stale:0 00010000:00010000:0.0:1782488722.049756:0:190286:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 7460 lines, 7460 kept, 0 dropped, 0 bad.