00000100:00100000:0.0F:1715207754.074205:0:3676:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:0.0:1715207754.074268:0:3676:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:0.0:1715207754.074300:0:3676:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:0.0:1715207754.074349:0:3676:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:1.0F:1715207754.074382:0:6554:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523505020416 00000100:00100000:0.0:1715207754.074390:0:3676:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:2.0F:1715207754.074405:0:7119:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523505020352 00000100:00100000:1.0:1715207754.074406:0:6554:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523505020416 00000100:00100000:2.0:1715207754.074424:0:7119:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523505020352 00000100:00100000:0.0:1715207754.074424:0:9386:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523505020608 00000100:00100000:0.0:1715207754.074435:0:9386:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523505020608 00000100:00100000:1.0:1715207754.074458:0:6554:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523505020544 00000100:00100000:1.0:1715207754.074461:0:6554:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523505020544 00000100:00100000:1.0:1715207754.074477:0:6554:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523505020480 00000100:00100000:1.0:1715207754.074479:0:6554:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523505020480 00000100:00100000:0.0:1715207754.074483:0:9386:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 2031 00000100:00100000:2.0:1715207754.074484:0:7119:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 1331 00000100:00100000:0.0:1715207754.074493:0:9386:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009e04e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+7:2002:x1798523505020608:12345-192.168.201.23@tcp:400:kworker.0 00000100:00100000:2.0:1715207754.074494:0:7119:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff880087dfd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:7e7a1b67-d841-4c3c-9f35-ded8f965b8dd+12:2003:x1798523505020352:12345-192.168.201.23@tcp:400:kworker.0 00000100:00100000:1.0:1715207754.074499:0:6554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 3817 00000100:00100000:1.0:1715207754.074507:0:6554:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88007e4ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:04532c2b-e5e9-420d-abb0-4480aab1e378+8:2002:x1798523505020416:12345-192.168.201.23@tcp:400:kworker.0 00000100:00100000:0.0:1715207754.074577:0:9386:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009e04e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+7:2002:x1798523505020608:12345-192.168.201.23@tcp:400:kworker.0 Request processed in 85us (185us total) trans 0 rc 0/0 00000100:00100000:0.0:1715207754.074586:0:9386:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 2031 00000100:00100000:2.0:1715207754.074591:0:7119:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff880087dfd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:7e7a1b67-d841-4c3c-9f35-ded8f965b8dd+12:2003:x1798523505020352:12345-192.168.201.23@tcp:400:kworker.0 Request processed in 99us (322us total) trans 0 rc 0/0 00000100:00100000:1.0:1715207754.074591:0:6554:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88007e4ec380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:04532c2b-e5e9-420d-abb0-4480aab1e378+8:2002:x1798523505020416:12345-192.168.201.23@tcp:400:kworker.0 Request processed in 84us (398us total) trans 0 rc 0/0 00000100:00100000:2.0:1715207754.074602:0:7119:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 1331 00000100:00100000:1.0:1715207754.074602:0:6554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 3817 00000100:00100000:1.0:1715207754.074621:0:6554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 3818 00000100:00100000:1.0:1715207754.074626:0:6554:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88007e4edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+9:2002:x1798523505020544:12345-192.168.201.23@tcp:400:kworker.0 00000100:00100000:2.0:1715207754.074645:0:9407:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 3819 00000100:00100000:1.0:1715207754.074647:0:6554:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88007e4edc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+9:2002:x1798523505020544:12345-192.168.201.23@tcp:400:kworker.0 Request processed in 21us (347us total) trans 0 rc 0/0 00000100:00100000:2.0:1715207754.074651:0:9407:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88007e4ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+20:2003:x1798523505020480:12345-192.168.201.23@tcp:400:kworker.0 00000100:00100000:1.0:1715207754.074655:0:6554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 3818 00000100:00080000:1.0:1715207754.074676:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 older than 1715207724 00000100:00080000:1.0:1715207754.074681:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 older than 1715207724 00000100:00100000:2.0:1715207754.074682:0:9407:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88007e4ee680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+20:2003:x1798523505020480:12345-192.168.201.23@tcp:400:kworker.0 Request processed in 31us (334us total) trans 0 rc 0/0 00000100:00080000:1.0:1715207754.074684:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd MGS older than 1715207724 00000100:00080000:1.0:1715207754.074686:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 older than 1715207724 00000100:00100000:2.0:1715207754.074690:0:9407:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 3819 00010000:00010000:2.0:1715207754.744103:0:7112:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1715207754.820153:0:3683:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88012d0afb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3683:1798523510603456:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1715207754.820177:0:3684:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88009e04ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3684:1798523510603520:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1715207754.820247:0:3683:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1715207754.820247:0:3684:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0F:1715207754.820355:0:8276:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510603520 00000100:00100000:0.0:1715207754.820361:0:9389:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510603456 00000100:00100000:0.0:1715207754.820376:0:9389:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510603456 00000100:00100000:3.0:1715207754.820381:0:8276:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510603520 00000100:00100000:3.0:1715207754.820430:0:8276:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1691 00000100:00100000:3.0:1715207754.820439:0:8276:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88012f838000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3684:x1798523510603520:12345-0@lo:41:osp-pre-1-0.0 00000100:00080000:1.0:1715207754.820446:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 older than 1715207724 00000100:00100000:0.0:1715207754.820477:0:9389:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1628 00000004:00000024:3.0:1715207754.820486:0:8276:0:(mdt_handler.c:569:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446309888 avail 1316847616 00000100:00100000:0.0:1715207754.820491:0:9389:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88012c8dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3683:x1798523510603456:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1715207754.820542:0:9389:0:(ofd_obd.c:743:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3922501632 avail 3690979328 00000100:00100000:3.0:1715207754.820567:0:8276:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88012f838000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3684:x1798523510603520:12345-0@lo:41:osp-pre-1-0.0 Request processed in 125us (317us total) trans 0 rc 0/0 00000100:00100000:3.0:1715207754.820579:0:8276:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1691 00000100:00100000:1.0:1715207754.820600:0:3684:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88009e04ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3684:1798523510603520:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1715207754.820672:0:9389:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88012c8dbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3683:x1798523510603456:12345-0@lo:13:osp-pre-0-0.0 Request processed in 181us (423us total) trans 0 rc 0/0 00000100:00100000:0.0:1715207754.820687:0:9389:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1628 00000100:00100000:2.0:1715207754.820721:0:3683:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88012d0afb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3683:1798523510603456:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1715207754.836142:0:3685:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88012d0aca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3685:1798523510603584:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1715207754.836196:0:3685:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1715207754.836283:0:9389:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510603584 00000100:00100000:2.0:1715207754.836314:0:9389:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510603584 00000100:00080000:1.0:1715207754.836364:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 older than 1715207724 00000100:00100000:2.0:1715207754.836372:0:9389:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1629 00000100:00100000:2.0:1715207754.836383:0:9389:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009e04f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3685:x1798523510603584:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1715207754.836411:0:9389:0:(ofd_obd.c:743:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3922501632 avail 3690979328 00000100:00100000:2.0:1715207754.836489:0:9389:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009e04f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3685:x1798523510603584:12345-0@lo:13:osp-pre-0-1.0 Request processed in 107us (291us total) trans 0 rc 0/0 00000100:00100000:2.0:1715207754.836498:0:9389:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1629 00000100:00100000:0.0:1715207754.836567:0:3685:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88012d0aca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3685:1798523510603584:0@lo:13:osp-pre-0-1.0 00000001:02000400:1.0:1715207755.243866:0:12452:0:(debug.c:705:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity-hsm test_12s: @@@@@@ FAIL: archive of /mnt/lustre/d12s.sanity-hsm/f12s.sanity-hsm failed 00000001:00080000:0.0:1715207755.461817:0:7105:0:(tgt_lastrcvd.c:969:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967414 is committed 00010000:00010000:2.0:1715207755.746529:0:11642:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 75 lines, 75 kept, 0 dropped, 0 bad.