00000100:00100000:2.0F:1715208105.276405:0:3215:0:(ptlrpcd.c:417:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0F:1715208105.276417:0:3217:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88013239bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:47336e3e-5c2f-4892-9b72-90c87ec36b26:3217:1798523508214464:0@lo:400:kworker.0 00000100:00100000:1.0F:1715208105.276418:0:3216:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88008e93c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3216:1798523508214656:0@lo:400:kworker.0 00000100:00100000:2.0:1715208105.276420:0:3215:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88008e93dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3215:1798523508214592:0@lo:400:kworker.0 00000100:00100000:2.0:1715208105.276479:0:3215:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1715208105.276479:0:3217:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1715208105.276492:0:3218:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff880132398a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3218:1798523508214528:0@lo:400:kworker.0 00000100:00100000:2.0:1715208105.276495:0:3215:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff880132399880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3215:1798523508214400:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1715208105.276495:0:3217:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88008e93f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3217:1798523508214720:0@lo:400:kworker.0 00000100:00100000:0.0:1715208105.276512:0:3217:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1715208105.276515:0:3218:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1715208105.276527:0:3215:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1715208105.276530:0:14590:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523508214720 00000100:00100000:0.0:1715208105.276545:0:14590:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523508214720 00000100:00100000:2.0:1715208105.276559:0:8623:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523508214592 00000100:00100000:1.0:1715208105.276560:0:8579:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523508214400 00000100:00100000:2.0:1715208105.276562:0:8623:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523508214592 00000100:00100000:1.0:1715208105.276565:0:8579:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523508214400 00000100:00080000:3.0F:1715208105.276573:0:6994:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 older than 1715208075 00000100:00100000:2.0:1715208105.276578:0:8623:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523508214528 00000100:00100000:0.0:1715208105.276578:0:14590:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1076 00000100:00100000:2.0:1715208105.276580:0:8623:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523508214528 00000100:00100000:0.0:1715208105.276582:0:14590:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88008e93d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3217:x1798523508214720:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1715208105.276584:0:8579:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 962 00000100:00080000:3.0:1715208105.276585:0:6994:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 older than 1715208075 00000100:00100000:2.0:1715208105.276589:0:8623:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2038 00000100:00100000:1.0:1715208105.276589:0:8579:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff8800902a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3215:x1798523508214400:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1715208105.276592:0:8623:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff8800902a2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3215:x1798523508214592:12345-0@lo:400:kworker.0 00002000:00000024:1.0:1715208105.276614:0:8579:0:(ofd_obd.c:743:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1715208105.276617:0:11480:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2039 00000100:00100000:0.0:1715208105.276619:0:14590:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88008e93d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3217:x1798523508214720:12345-0@lo:400:kworker.0 Request processed in 34us (103us total) trans 0 rc 0/0 00000100:00100000:0.0:1715208105.276626:0:14590:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1076 00000100:00100000:3.0:1715208105.276631:0:11480:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88008e6e3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3218:x1798523508214528:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1715208105.276631:0:8623:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff8800902a2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3215:x1798523508214592:12345-0@lo:400:kworker.0 Request processed in 37us (153us total) trans 0 rc 0/0 00000100:00100000:2.0:1715208105.276635:0:8623:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2038 00000100:00100000:1.0:1715208105.276642:0:8579:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff8800902a1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3215:x1798523508214400:12345-0@lo:13:osp-pre-1-0.0 Request processed in 52us (113us total) trans 0 rc 0/0 00000100:00100000:1.0:1715208105.276648:0:8579:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 962 00000100:00100000:2.0:1715208105.276650:0:6992:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523508214464 00000100:00100000:2.0:1715208105.276651:0:6992:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523508214464 00000100:00100000:0.0:1715208105.276664:0:3217:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88008e93f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3217:1798523508214720:0@lo:400:kworker.0 00000100:00100000:1.0:1715208105.276669:0:3216:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1715208105.276681:0:6992:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1346 00000100:00100000:2.0:1715208105.276684:0:6992:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88012cb15c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:47336e3e-5c2f-4892-9b72-90c87ec36b26+18:3217:x1798523508214464:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1715208105.276690:0:8623:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523508214656 00000100:00100000:0.0:1715208105.276692:0:8623:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523508214656 00000100:00100000:0.0:1715208105.276706:0:8623:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2040 00000100:00100000:2.0:1715208105.276707:0:6992:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88012cb15c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:47336e3e-5c2f-4892-9b72-90c87ec36b26+18:3217:x1798523508214464:12345-0@lo:400:kworker.0 Request processed in 22us (229us total) trans 0 rc 0/0 00000100:00100000:0.0:1715208105.276708:0:8623:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88008e6e2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3216:x1798523508214656:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1715208105.276711:0:6992:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1346 00000100:00100000:3.0:1715208105.276723:0:11480:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88008e6e3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3218:x1798523508214528:12345-0@lo:400:kworker.0 Request processed in 96us (205us total) trans 0 rc 0/0 00000100:00100000:0.0:1715208105.276731:0:8623:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88008e6e2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3216:x1798523508214656:12345-0@lo:400:kworker.0 Request processed in 21us (62us total) trans 0 rc 0/0 00000100:00100000:3.0:1715208105.276732:0:11480:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2039 00000100:00100000:2.0:1715208105.276732:0:3215:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88008e93dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3215:1798523508214592:0@lo:400:kworker.0 00000100:00100000:0.0:1715208105.276736:0:8623:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2040 00000100:00100000:1.0:1715208105.276741:0:3218:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff880132398a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3218:1798523508214528:0@lo:400:kworker.0 00000100:00100000:0.0:1715208105.276749:0:3217:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88013239bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:47336e3e-5c2f-4892-9b72-90c87ec36b26:3217:1798523508214464:0@lo:400:kworker.0 00000100:00100000:2.0:1715208105.276755:0:3215:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff880132399880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3215:1798523508214400:0@lo:13:osp-pre-1-0.0 00000100:00080000:3.0:1715208105.276761:0:6994:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 older than 1715208075 00000100:00100000:1.0:1715208105.276763:0:3216:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88008e93c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3216:1798523508214656:0@lo:400:kworker.0 00000100:00080000:3.0:1715208105.276764:0:6994:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd MGS older than 1715208075 00000100:00100000:0.0:1715208105.725041:0:3209:0:(events.c:359:request_in_callback()) peer: 12345-192.168.202.38@tcp (source: 12345-192.168.202.38@tcp) 00000100:00100000:3.0:1715208105.725126:0:11480:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523503950400 00000100:00100000:3.0:1715208105.725145:0:11480:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523503950400 00000100:00080000:2.0:1715208105.725227:0:6994:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 older than 1715208075 00000100:00100000:3.0:1715208105.725240:0:11480:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.38@tcp, seq: 405 00000100:00100000:3.0:1715208105.725250:0:11480:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88008e93f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:cbae74da-7571-45ad-b971-0e0378bbd4e4+43:19186:x1798523503950400:12345-192.168.202.38@tcp:60:lhsmtool_posix.0 00000100:00100000:3.0:1715208105.725404:0:11480:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88008e93f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:cbae74da-7571-45ad-b971-0e0378bbd4e4+43:19186:x1798523503950400:12345-192.168.202.38@tcp:60:lhsmtool_posix.0 Request processed in 156us (372us total) trans 0 rc 0/0 00000100:00100000:3.0:1715208105.725411:0:11480:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.38@tcp, seq: 405 00000100:00100000:1.0:1715208105.726948:0:3210:0:(events.c:359:request_in_callback()) peer: 12345-192.168.202.38@tcp (source: 12345-192.168.202.38@tcp) 00000100:00100000:2.0:1715208105.727006:0:13448:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523503950528 00000100:00100000:2.0:1715208105.727016:0:13448:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523503950528 00000100:00100000:2.0:1715208105.727063:0:13448:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.38@tcp, seq: 86 00000100:00100000:2.0:1715208105.727070:0:13448:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88008e6e3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:cbae74da-7571-45ad-b971-0e0378bbd4e4+43:19186:x1798523503950528:12345-192.168.202.38@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1715208105.727384:0:13448:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88008e6e3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:cbae74da-7571-45ad-b971-0e0378bbd4e4+44:19186:x1798523503950528:12345-192.168.202.38@tcp:35:lhsmtool_posix.0 Request processed in 312us (438us total) trans 4294967502 rc 0/0 00000100:00100000:2.0:1715208105.727393:0:13448:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.38@tcp, seq: 86 00010000:00010000:2.0:1715208105.826581:0:10149:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1715208106.828543:0:18958:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1715208107.150450:0:7244:0:(tgt_lastrcvd.c:969:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967502 is committed 00000040:00080000:0.0:1715208107.236450:0:14992:0:(llog.c:735:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1715208107.236457:0:14992:0:(llog_cat.c:796:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1715208107.237021:0:14992:0:(llog.c:792:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1715208107.237029:0:14992:0:(llog.c:792:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1715208107.278398:0:3208:0:(events.c:359:request_in_callback()) peer: 12345-192.168.202.38@tcp (source: 12345-192.168.202.38@tcp) 00000100:00100000:0.0:1715208107.278441:0:3208:0:(events.c:359:request_in_callback()) peer: 12345-192.168.202.38@tcp (source: 12345-192.168.202.38@tcp) 00000100:00100000:0.0:1715208107.278492:0:3208:0:(events.c:359:request_in_callback()) peer: 12345-192.168.202.38@tcp (source: 12345-192.168.202.38@tcp) 00000100:00100000:2.0:1715208107.278498:0:6992:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523503950592 00000100:00100000:2.0:1715208107.278512:0:6992:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523503950592 00000100:00100000:3.0:1715208107.278585:0:11480:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523503950656 00000100:00100000:3.0:1715208107.278594:0:11480:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523503950656 00000100:00100000:1.0:1715208107.278603:0:9149:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523503950720 00000100:00100000:1.0:1715208107.278612:0:9149:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523503950720 00000100:00100000:2.0:1715208107.278616:0:6992:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.38@tcp, seq: 1347 00000100:00100000:2.0:1715208107.278627:0:6992:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88012cb15850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ff645b43-0d7b-4a37-b9cc-6ac8f8d65c0d+12:1956:x1798523503950592:12345-192.168.202.38@tcp:400:kworker.0 00000100:00100000:3.0:1715208107.278639:0:11480:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.38@tcp, seq: 2041 00000100:00100000:3.0:1715208107.278646:0:11480:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88008e93f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:cbae74da-7571-45ad-b971-0e0378bbd4e4+43:1957:x1798523503950656:12345-192.168.202.38@tcp:400:kworker.0 00000100:00100000:1.0:1715208107.278654:0:9149:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.38@tcp, seq: 1077 00000100:00100000:1.0:1715208107.278660:0:9149:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88008e93c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:cbae74da-7571-45ad-b971-0e0378bbd4e4+12:1956:x1798523503950720:12345-192.168.202.38@tcp:400:kworker.0 00000100:00080000:0.0:1715208107.278674:0:6994:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 older than 1715208077 00000100:00080000:0.0:1715208107.278679:0:6994:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 older than 1715208077 00000100:00080000:0.0:1715208107.278681:0:6994:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd MGS older than 1715208077 00000100:00100000:2.0:1715208107.278710:0:6992:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88012cb15850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:ff645b43-0d7b-4a37-b9cc-6ac8f8d65c0d+12:1956:x1798523503950592:12345-192.168.202.38@tcp:400:kworker.0 Request processed in 82us (318us total) trans 0 rc 0/0 00000100:00100000:3.0:1715208107.278712:0:11480:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88008e93f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:cbae74da-7571-45ad-b971-0e0378bbd4e4+43:1957:x1798523503950656:12345-192.168.202.38@tcp:400:kworker.0 Request processed in 66us (271us total) trans 0 rc 0/0 00000100:00100000:1.0:1715208107.278715:0:9149:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88008e93c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:cbae74da-7571-45ad-b971-0e0378bbd4e4+12:1956:x1798523503950720:12345-192.168.202.38@tcp:400:kworker.0 Request processed in 55us (224us total) trans 0 rc 0/0 00000100:00100000:3.0:1715208107.278721:0:11480:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.38@tcp, seq: 2041 00000100:00100000:2.0:1715208107.278722:0:6992:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.38@tcp, seq: 1347 00000100:00100000:1.0:1715208107.278723:0:9149:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.38@tcp, seq: 1077 00010000:00010000:2.0:1715208107.830474:0:6987:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1715208108.832583:0:8591:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1715208109.834387:0:6986:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1715208110.076462:0:3217:0:(ptlrpcd.c:417:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1715208110.076490:0:3217:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88008e93dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3217:1798523508214784:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1715208110.076597:0:3217:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1715208110.076631:0:8579:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523508214784 00000100:00100000:1.0:1715208110.076638:0:8579:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523508214784 00000100:00080000:0.0:1715208110.076653:0:6994:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 older than 1715208080 00000100:00100000:1.0:1715208110.076666:0:8579:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 963 00000100:00100000:1.0:1715208110.076671:0:8579:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88008f55d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3217:x1798523508214784:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1715208110.076702:0:8579:0:(ofd_obd.c:743:ofd_statfs()) blocks cached 0 granted 54460416 pending 0 free 3851419648 avail 3849322496 00000100:00100000:1.0:1715208110.076757:0:8579:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88008f55d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3217:x1798523508214784:12345-0@lo:13:osp-pre-0-0.0 Request processed in 83us (158us total) trans 0 rc 0/0 00000100:00100000:1.0:1715208110.076766:0:8579:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 963 00000100:00100000:0.0:1715208110.076815:0:3217:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88008e93dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3217:1798523508214784:0@lo:13:osp-pre-0-0.0 00000001:02000400:3.0:1715208110.248986:0:27585:0:(debug.c:705:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 25a: Restore lost file (HS_LOST flag) from import ========================================================== 18:41:49 (1715208109) 00000100:00100000:0.0:1715208110.284372:0:3217:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800902a2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3217:1798523508214976:0@lo:400:kworker.0 00000100:00100000:2.0:1715208110.284383:0:3218:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800902a3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3218:1798523508215040:0@lo:400:kworker.0 00000100:00100000:3.0:1715208110.284409:0:3215:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88008f55f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3215:1798523508214912:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1715208110.284416:0:3217:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1715208110.284433:0:11480:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523508214976 00000100:00100000:3.0:1715208110.284437:0:3215:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1715208110.284438:0:11480:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523508214976 00000100:00100000:3.0:1715208110.284455:0:3215:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800902a0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3215:1798523508215104:0@lo:400:kworker.0 00000100:00100000:2.0:1715208110.284465:0:11480:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2042 00000100:00100000:3.0:1715208110.284466:0:3215:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1715208110.284469:0:11480:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88008f55d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3217:x1798523508214976:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1715208110.284470:0:8623:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523508215104 00000100:00100000:0.0:1715208110.284474:0:8623:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523508215104 00000100:00100000:3.0:1715208110.284475:0:3215:0:(ptlrpcd.c:417:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1715208110.284478:0:3215:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800902a0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:47336e3e-5c2f-4892-9b72-90c87ec36b26:3215:1798523508214848:0@lo:400:kworker.0 00000100:00100000:0.0:1715208110.284490:0:8623:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2043 00000100:00100000:3.0:1715208110.284508:0:3215:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1715208110.284509:0:8623:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88008db11f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3215:x1798523508215104:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1715208110.284518:0:3215:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800902a1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3215:1798523508215168:0@lo:400:kworker.0 00000100:00100000:2.0:1715208110.284521:0:11480:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88008f55d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3217:x1798523508214976:12345-0@lo:400:kworker.0 Request processed in 51us (104us total) trans 0 rc 0/0 00000100:00100000:2.0:1715208110.284525:0:11480:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2042 00000100:00100000:3.0:1715208110.284531:0:3215:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1715208110.284533:0:8623:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88008db11f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3215:x1798523508215104:12345-0@lo:400:kworker.0 Request processed in 26us (67us total) trans 0 rc 0/0 00000100:00100000:0.0:1715208110.284538:0:8623:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2043 00000100:00100000:2.0:1715208110.284542:0:6992:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523508214848 00000100:00100000:2.0:1715208110.284544:0:6992:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523508214848 00000100:00100000:3.0:1715208110.284550:0:3215:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800902a0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3215:1798523508215104:0@lo:400:kworker.0 00000100:00080000:0.0:1715208110.284550:0:6994:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd MGS older than 1715208080 00000100:00080000:0.0:1715208110.284552:0:6994:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 older than 1715208080 00000100:00100000:2.0:1715208110.284556:0:6992:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1348 00000100:00100000:2.0:1715208110.284564:0:6992:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88012cb15450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:47336e3e-5c2f-4892-9b72-90c87ec36b26+18:3215:x1798523508214848:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1715208110.284569:0:8579:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523508214912 00000100:00100000:3.0:1715208110.284572:0:8579:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523508214912 00000100:00100000:2.0:1715208110.284583:0:6992:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88012cb15450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:47336e3e-5c2f-4892-9b72-90c87ec36b26+18:3215:x1798523508214848:12345-0@lo:400:kworker.0 Request processed in 20us (91us total) trans 0 rc 0/0 00000100:00080000:0.0:1715208110.284586:0:6994:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 older than 1715208080 00000100:00100000:2.0:1715208110.284587:0:6992:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1348 00000100:00100000:3.0:1715208110.284599:0:8579:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 964 00000100:00100000:2.0:1715208110.284601:0:3218:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1715208110.284603:0:8579:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88008db13800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3215:x1798523508214912:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1715208110.284610:0:6996:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523508215040 00000100:00100000:0.0:1715208110.284614:0:6996:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523508215040 00002000:00000024:3.0:1715208110.284619:0:8579:0:(ofd_obd.c:743:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1715208110.284621:0:3217:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800902a2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3217:1798523508214976:0@lo:400:kworker.0 00000100:00100000:0.0:1715208110.284632:0:6996:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2044 00000100:00100000:3.0:1715208110.284636:0:8579:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88008db13800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3215:x1798523508214912:12345-0@lo:13:osp-pre-1-0.0 Request processed in 33us (199us total) trans 0 rc 0/0 00000100:00100000:0.0:1715208110.284637:0:6996:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff8800902a1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3218:x1798523508215040:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1715208110.284640:0:8579:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 964 00000100:00100000:3.0:1715208110.284651:0:9149:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523508215168 00000100:00100000:3.0:1715208110.284653:0:9149:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523508215168 00000100:00100000:3.0:1715208110.284665:0:9149:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1078 00000100:00100000:3.0:1715208110.284668:0:9149:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88008db13b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3215:x1798523508215168:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1715208110.284668:0:6996:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff8800902a1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3218:x1798523508215040:12345-0@lo:400:kworker.0 Request processed in 30us (65us total) trans 0 rc 0/0 00000100:00100000:2.0:1715208110.284674:0:3218:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800902a3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3218:1798523508215040:0@lo:400:kworker.0 00000100:00100000:3.0:1715208110.284681:0:9149:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88008db13b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3215:x1798523508215168:12345-0@lo:400:kworker.0 Request processed in 12us (149us total) trans 0 rc 0/0 00000100:00100000:0.0:1715208110.284682:0:6996:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2044 00000100:00100000:3.0:1715208110.284684:0:9149:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1078 00000100:00100000:3.0:1715208110.284695:0:3215:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800902a0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:47336e3e-5c2f-4892-9b72-90c87ec36b26:3215:1798523508214848:0@lo:400:kworker.0 00000100:00080000:0.0:1715208110.284696:0:6994:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 older than 1715208080 00000100:00100000:3.0:1715208110.284701:0:3215:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800902a1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3215:1798523508215168:0@lo:400:kworker.0 00000100:00100000:3.0:1715208110.284715:0:3215:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88008f55f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3215:1798523508214912:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1715208110.305565:0:3209:0:(events.c:359:request_in_callback()) peer: 12345-192.168.202.38@tcp (source: 12345-192.168.202.38@tcp) 00000100:00100000:3.0:1715208110.305614:0:6997:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523503950912 00000100:00100000:3.0:1715208110.305620:0:6997:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523503950912 00000100:00080000:0.0:1715208110.305670:0:6994:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 older than 1715208080 00000100:00100000:3.0:1715208110.305671:0:6997:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.38@tcp, seq: 406 00000100:00100000:3.0:1715208110.305676:0:6997:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88008f55f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:cbae74da-7571-45ad-b971-0e0378bbd4e4+43:20567:x1798523503950912:12345-192.168.202.38@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1715208110.305686:0:6997:0:(ldlm_lockd.c:1317:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1715208110.305700:0:6997:0:(ldlm_lockd.c:1389:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff88012ffd7a80/0x8c55e4ad37f6703e lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xbbeffb005ec3e854 expref: -99 pid: 6997 timeout: 0 lvb_type: 0 00000004:00080000:3.0:1715208110.305987:0:6997:0:(mdt_identity.c:116:mdt_identity_do_upcall()) lustre-MDT0000: invoked upcall /home/green/git/lustre-release/lustre/utils/l_getidentity lustre-MDT0000 0, time 253us 00010000:00010000:3.0:1715208110.308071:0:6997:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff88012ffd5440/0x8c55e4ad37f67045 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6997 initiator: MDT0 00010000:00010000:3.0:1715208110.308083:0:6997:0:(ldlm_lock.c:1089:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff88012ffd5440/0x8c55e4ad37f67045 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6997 initiator: MDT0 00010000:00010000:3.0:1715208110.308092:0:6997:0:(ldlm_request.c:538:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff88012ffd5440/0x8c55e4ad37f67045 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6997 initiator: MDT0 00010000:00010000:3.0:1715208110.308100:0:6997:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff88012ffd6880/0x8c55e4ad37f6704c lrc: 3/1,0 mode: --/PR res: [0x200000002:0x1:0x0].0x646966 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6997 initiator: MDT0 00010000:00010000:3.0:1715208110.308104:0:6997:0:(ldlm_lock.c:1089:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff88012ffd6880/0x8c55e4ad37f6704c lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x646966 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6997 initiator: MDT0 00010000:00010000:3.0:1715208110.308107:0:6997:0:(ldlm_request.c:538:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff88012ffd6880/0x8c55e4ad37f6704c lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x646966 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6997 initiator: MDT0 00000004:00200000:3.0:1715208110.308160:0:6997:0:(mdt_handler.c:938:mdt_pack_attr2body()) [0x200000002:0x2:0x0]: returning size 10752 00010000:00010000:3.0:1715208110.308254:0:6997:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff88012ffd5440/0x8c55e4ad37f67045 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6997 initiator: MDT0 00010000:00010000:3.0:1715208110.308264:0:6997:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff88012ffd5440/0x8c55e4ad37f67045 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6997 initiator: MDT0 00010000:00010000:3.0:1715208110.308288:0:6997:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff88012ffd5440/0x8c55e4ad37f67045 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6997 initiator: MDT0 00010000:00010000:3.0:1715208110.308292:0:6997:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff88012ffd5440 already unused, calling callback (ffffffffa13a3730) 00010000:00010000:3.0:1715208110.308295:0:6997:0:(ldlm_request.c:374:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff88012ffd5440/0x8c55e4ad37f67045 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6997 initiator: MDT0 00010000:00010000:3.0:1715208110.308299:0:6997:0:(ldlm_request.c:1325:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff88012ffd5440/0x8c55e4ad37f67045 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6997 initiator: MDT0 00010000:00010000:3.0:1715208110.308307:0:6997:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x0 (ffff88008e524300) 00010000:00010000:3.0:1715208110.308309:0:6997:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1715208110.308311:0:6997:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff88012ffd5440/0x8c55e4ad37f67045 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6997 initiator: MDT0 00010000:00010000:3.0:1715208110.308314:0:6997:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff88012ffd5440/0x8c55e4ad37f67045 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6997 initiator: MDT0 00010000:00010000:3.0:1715208110.308329:0:6997:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff88012ffd6880/0x8c55e4ad37f6704c lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x646966 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6997 initiator: MDT0 00010000:00010000:3.0:1715208110.308333:0:6997:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff88012ffd6880/0x8c55e4ad37f6704c lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x646966 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6997 initiator: MDT0 00010000:00010000:3.0:1715208110.308336:0:6997:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff88012ffd6880/0x8c55e4ad37f6704c lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x646966 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6997 initiator: MDT0 00010000:00010000:3.0:1715208110.308338:0:6997:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff88012ffd6880 already unused, calling callback (ffffffffa13a3730) 00010000:00010000:3.0:1715208110.308340:0:6997:0:(ldlm_request.c:374:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff88012ffd6880/0x8c55e4ad37f6704c lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x646966 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6997 initiator: MDT0 00010000:00010000:3.0:1715208110.308348:0:6997:0:(ldlm_request.c:1325:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff88012ffd6880/0x8c55e4ad37f6704c lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x646966 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6997 initiator: MDT0 00010000:00010000:3.0:1715208110.308353:0:6997:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x646966 (ffff88008febb100) 00010000:00010000:3.0:1715208110.308354:0:6997:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1715208110.308355:0:6997:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff88012ffd6880/0x8c55e4ad37f6704c lrc: 2/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x646966 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6997 initiator: MDT0 00010000:00010000:3.0:1715208110.308358:0:6997:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff88012ffd6880/0x8c55e4ad37f6704c lrc: 0/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x646966 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6997 initiator: MDT0 00010000:00010000:3.0:1715208110.308382:0:6997:0:(ldlm_lockd.c:1559:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff88012ffd7a80/0x8c55e4ad37f6703e lrc: 1/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.38@tcp remote: 0xbbeffb005ec3e854 expref: 45 pid: 6997 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715208110.308387:0:6997:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff88012ffd7a80/0x8c55e4ad37f6703e lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.38@tcp remote: 0xbbeffb005ec3e854 expref: 45 pid: 6997 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1715208110.308391:0:6997:0:(ldlm_lockd.c:1637:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff88012ffd7a80, rc 0) 00000100:00100000:3.0:1715208110.308425:0:6997:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88008f55f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:cbae74da-7571-45ad-b971-0e0378bbd4e4+44:20567:x1798523503950912:12345-192.168.202.38@tcp:101:lhsmtool_posix.0 Request processed in 2749us (2865us total) trans 4294967503 rc 301/301 00000100:00100000:3.0:1715208110.308430:0:6997:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.38@tcp, seq: 406 00000100:00100000:0.0:1715208110.309297:0:3210:0:(events.c:359:request_in_callback()) peer: 12345-192.168.202.38@tcp (source: 12345-192.168.202.38@tcp) 00000100:00100000:3.0:1715208110.309339:0:6997:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523503950976 00000100:00100000:3.0:1715208110.309344:0:6997:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523503950976 00000100:00100000:3.0:1715208110.309367:0:6997:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.38@tcp, seq: 407 00000100:00100000:3.0:1715208110.309371:0:6997:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88008e93dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:cbae74da-7571-45ad-b971-0e0378bbd4e4+44:20567:x1798523503950976:12345-192.168.202.38@tcp:59:lhsmtool_posix.0 00000100:00100000:3.0:1715208110.309410:0:6997:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88008e93dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:cbae74da-7571-45ad-b971-0e0378bbd4e4+44:20567:x1798523503950976:12345-192.168.202.38@tcp:59:lhsmtool_posix.0 Request processed in 39us (116us total) trans 0 rc 0/0 00000100:00100000:3.0:1715208110.309415:0:6997:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.38@tcp, seq: 407 00000001:02000400:1.0:1715208110.791095:0:27651:0:(debug.c:705:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity-hsm test_25a: @@@@@@ FAIL: cannot create directory '/mnt/lustre/d25a.sanity-hsm' 00010000:00010000:2.0:1715208110.836532:0:10149:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1715208111.236559:0:14992:0:(llog.c:735:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1715208111.236570:0:14992:0:(llog_cat.c:796:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1715208111.237258:0:14992:0:(llog.c:792:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1715208111.237270:0:14992:0:(llog.c:792:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 Debug log: 236 lines, 236 kept, 0 dropped, 0 bad.