00010000:00010000:2.0F:1715208049.714505:0:18958:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1715208049.980530:0:3218:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8801329f9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3218:1798523508209408:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1715208049.980610:0:3218:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0F:1715208049.980683:0:8579:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523508209408 00000100:00100000:3.0:1715208049.980704:0:8579:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523508209408 00000100:00080000:0.0F:1715208049.980763:0:6994:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 older than 1715208019 00000100:00100000:3.0:1715208049.980781:0:8579:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 939 00000100:00100000:3.0:1715208049.980790:0:8579:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff8800a7775c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3218:x1798523508209408:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1715208049.980830:0:8579:0:(ofd_obd.c:743:ofd_statfs()) blocks cached 0 granted 51052544 pending 0 free 3853516800 avail 3851419648 00000100:00100000:3.0:1715208049.980897:0:8579:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff8800a7775c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3218:x1798523508209408:12345-0@lo:13:osp-pre-0-0.0 Request processed in 104us (284us total) trans 0 rc 0/0 00000100:00100000:3.0:1715208049.980909:0:8579:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 939 00000100:00100000:2.0:1715208049.980962:0:3218:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8801329f9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3218:1798523508209408:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1715208050.188400:0:3217:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff880135a98380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3217:1798523508209600:0@lo:400:kworker.0 00000100:00100000:0.0:1715208050.188465:0:3217:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0F:1715208050.188473:0:3216:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff880135a9b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3216:1798523508209536:0@lo:400:kworker.0 00000100:00100000:0.0:1715208050.188496:0:3217:0:(ptlrpcd.c:417:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1715208050.188503:0:3217:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff880135a98700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3217:1798523508209664:0@lo:400:kworker.0 00000100:00100000:1.0:1715208050.188524:0:3216:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1715208050.188535:0:6998:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523508209600 00000100:00100000:0.0:1715208050.188542:0:6998:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523508209600 00000100:00100000:1.0:1715208050.188552:0:3216:0:(ptlrpcd.c:417:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1715208050.188559:0:3216:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff880135a9b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:47336e3e-5c2f-4892-9b72-90c87ec36b26:3216:1798523508209472:0@lo:400:kworker.0 00000100:00100000:0.0:1715208050.188577:0:6998:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523508209536 00000100:00100000:0.0:1715208050.188580:0:6998:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523508209536 00000100:00100000:1.0:1715208050.188583:0:3216:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1715208050.188594:0:3216:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff880135a9ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3216:1798523508209728:0@lo:400:kworker.0 00000100:00100000:0.0:1715208050.188596:0:6998:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1998 00000100:00100000:0.0:1715208050.188603:0:6998:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff880135a9aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3217:x1798523508209600:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1715208050.188612:0:3216:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1715208050.188630:0:6997:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1999 00000100:00100000:2.0:1715208050.188640:0:6997:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88008db74a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3216:x1798523508209536:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1715208050.188646:0:6992:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523508209472 00000100:00100000:0.0:1715208050.188648:0:6998:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff880135a9aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3217:x1798523508209600:12345-0@lo:400:kworker.0 Request processed in 44us (183us total) trans 0 rc 0/0 00000100:00100000:1.0:1715208050.188652:0:6992:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523508209472 00000100:00100000:0.0:1715208050.188656:0:6998:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1998 00000100:00080000:0.0:1715208050.188677:0:6994:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd MGS older than 1715208020 00000100:00100000:1.0:1715208050.188678:0:6992:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1312 00000100:00080000:0.0:1715208050.188681:0:6994:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 older than 1715208020 00000100:00100000:1.0:1715208050.188683:0:6992:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88012bdae450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:47336e3e-5c2f-4892-9b72-90c87ec36b26+18:3216:x1798523508209472:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1715208050.188691:0:3217:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1715208050.188695:0:6997:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88008db74a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3216:x1798523508209536:12345-0@lo:400:kworker.0 Request processed in 56us (171us total) trans 0 rc 0/0 00000100:00100000:2.0:1715208050.188703:0:6997:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1999 00000100:00100000:1.0:1715208050.188717:0:6992:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88012bdae450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:47336e3e-5c2f-4892-9b72-90c87ec36b26+18:3216:x1798523508209472:12345-0@lo:400:kworker.0 Request processed in 32us (132us total) trans 0 rc 0/0 00000100:00100000:2.0:1715208050.188720:0:6997:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523508209664 00000100:00100000:0.0:1715208050.188720:0:3217:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff880135a98380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3217:1798523508209600:0@lo:400:kworker.0 00000100:00100000:1.0:1715208050.188724:0:6992:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1312 00000100:00100000:2.0:1715208050.188725:0:6997:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523508209664 00000100:00100000:2.0:1715208050.188750:0:6997:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2000 00000100:00100000:2.0:1715208050.188754:0:6997:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff880135a99c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3217:x1798523508209664:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1715208050.188784:0:6997:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff880135a99c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3217:x1798523508209664:12345-0@lo:400:kworker.0 Request processed in 29us (92us total) trans 0 rc 0/0 00000100:00100000:2.0:1715208050.188791:0:6997:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2000 00000100:00080000:2.0:1715208050.188805:0:6994:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 older than 1715208020 00000100:00100000:2.0:1715208050.188826:0:3216:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff880135a9b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:47336e3e-5c2f-4892-9b72-90c87ec36b26:3216:1798523508209472:0@lo:400:kworker.0 00000100:00100000:2.0:1715208050.188839:0:3216:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff880135a9b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3216:1798523508209536:0@lo:400:kworker.0 00000100:00100000:2.0:1715208050.188858:0:3216:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8801329fb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3216:1798523508209792:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1715208050.188877:0:8575:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523508209728 00000100:00100000:0.0:1715208050.188882:0:3217:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff880135a98700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3217:1798523508209664:0@lo:400:kworker.0 00000100:00100000:2.0:1715208050.188884:0:3216:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1715208050.188886:0:8575:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523508209728 00000100:00080000:2.0:1715208050.188908:0:6994:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 older than 1715208020 00000100:00100000:3.0:1715208050.188927:0:8575:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1055 00000100:00100000:3.0:1715208050.188938:0:8575:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88008db77100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3216:x1798523508209728:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1715208050.189004:0:8575:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88008db77100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3216:x1798523508209728:12345-0@lo:400:kworker.0 Request processed in 66us (388us total) trans 0 rc 0/0 00000100:00100000:3.0:1715208050.189046:0:8575:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1055 00000100:00100000:2.0:1715208050.189089:0:8579:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523508209792 00000100:00100000:2.0:1715208050.189095:0:8579:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523508209792 00000100:00100000:2.0:1715208050.189122:0:8579:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 940 00000100:00100000:2.0:1715208050.189127:0:8579:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff8800a7777480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3216:x1798523508209792:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1715208050.189166:0:8579:0:(ofd_obd.c:743:ofd_statfs()) blocks cached 0 granted 40828928 pending 0 free 3857711104 avail 3855613952 00000100:00100000:2.0:1715208050.189200:0:8579:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff8800a7777480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3216:x1798523508209792:12345-0@lo:13:osp-pre-1-0.0 Request processed in 71us (315us total) trans 0 rc 0/0 00000100:00100000:2.0:1715208050.189207:0:8579:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 940 00000100:00100000:2.0:1715208050.189243:0:3216:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8801329fb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3216:1798523508209792:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1715208050.189253:0:3216:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff880135a9ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3216:1798523508209728:0@lo:400:kworker.0 00000001:02000400:1.0:1715208050.701387:0:16721:0:(debug.c:705:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity-hsm test_23: @@@@@@ FAIL: hsm flags on /mnt/lustre/d23.sanity-hsm/test_mtime are Dumping lctl log to /tmp/testlogs//sanity-hsm.test_23.*.1715208049.log != 0x00000000 00010000:00010000:2.0:1715208050.716425:0:6987:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 75 lines, 75 kept, 0 dropped, 0 bad.