00010000:00010000:2.0F:1715207120.472090:0:11642:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1715207120.660040:0:3684:0:(ptlrpcd.c:417:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1715207120.660063:0:3684:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a0392d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3684:1798523510448448:0@lo:400:kworker.0 00000100:00100000:0.0F:1715207120.660088:0:3683:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a0392a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3683:1798523510448640:0@lo:400:kworker.0 00000100:00100000:3.0F:1715207120.660114:0:3685:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a0390000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3685:1798523510448512:0@lo:400:kworker.0 00000100:00100000:2.0:1715207120.660200:0:3684:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1715207120.660227:0:3683:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1715207120.660239:0:3684:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88009e0c8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3684:1798523510448384:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1715207120.660273:0:7122:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510448448 00000100:00100000:2.0:1715207120.660277:0:3684:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1715207120.660284:0:7122:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510448448 00000100:00100000:0.0:1715207120.660289:0:3683:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88012f5cb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3683:1798523510448896:0@lo:400:kworker.0 00000100:00100000:2.0:1715207120.660295:0:3684:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff880139bf2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3684:1798523510448704:0@lo:400:kworker.0 00000100:00100000:2.0:1715207120.660312:0:3684:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0F:1715207120.660318:0:8276:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510448640 00000100:00100000:2.0:1715207120.660325:0:3684:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88012d60ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3684:1798523510448960:0@lo:400:kworker.0 00000100:00100000:1.0:1715207120.660332:0:8276:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510448640 00000100:00100000:3.0:1715207120.660335:0:7122:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510448704 00000100:00100000:3.0:1715207120.660339:0:7122:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510448704 00000100:00100000:2.0:1715207120.660340:0:3684:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1715207120.660353:0:3684:0:(ptlrpcd.c:417:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1715207120.660358:0:3684:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88012ef0a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3684:1798523510449152:0@lo:400:kworker.0 00000100:00100000:0.0:1715207120.660365:0:3683:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1715207120.660367:0:7122:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510448960 00000100:00100000:1.0:1715207120.660367:0:8276:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1171 00000100:00100000:3.0:1715207120.660370:0:7122:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510448960 00000100:00100000:1.0:1715207120.660379:0:8276:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88012c8e2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3683:x1798523510448640:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1715207120.660382:0:9389:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510448384 00000100:00100000:2.0:1715207120.660387:0:9389:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510448384 00000100:00100000:3.0:1715207120.660390:0:7122:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2599 00000100:00100000:3.0:1715207120.660397:0:7122:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009e0c9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3684:x1798523510448448:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1715207120.660421:0:3686:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a0392300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:24597633-7c6c-43cc-a7e8-ad821c1018c0:3686:1798523510448320:0@lo:400:kworker.0 00000100:00100000:2.0:1715207120.660423:0:9389:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1122 00000100:00100000:2.0:1715207120.660430:0:9389:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009e0c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3684:x1798523510448384:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1715207120.660456:0:7122:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009e0c9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3684:x1798523510448448:12345-0@lo:400:kworker.0 Request processed in 55us (254us total) trans 0 rc 0/0 00000100:00100000:1.0:1715207120.660456:0:8276:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88012c8e2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3683:x1798523510448640:12345-0@lo:400:kworker.0 Request processed in 77us (229us total) trans 0 rc 0/0 00002000:00000024:2.0:1715207120.660457:0:9389:0:(ofd_obd.c:743:ofd_statfs()) blocks cached 0 granted 16080576 pending 0 free 3921453056 avail 3689930752 00000100:00100000:3.0:1715207120.660468:0:7122:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2599 00000100:00100000:1.0:1715207120.660468:0:8276:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1171 00000100:00100000:0.0:1715207120.660469:0:3686:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1715207120.660492:0:7122:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2600 00000100:00100000:2.0:1715207120.660492:0:9389:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009e0c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3684:x1798523510448384:12345-0@lo:13:osp-pre-1-0.0 Request processed in 61us (213us total) trans 0 rc 0/0 00000100:00100000:0.0:1715207120.660493:0:3686:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a0390a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3686:1798523510448576:0@lo:400:kworker.0 00000100:00100000:3.0:1715207120.660499:0:7122:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009e140e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3684:x1798523510448704:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1715207120.660500:0:9389:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1122 00000100:00100000:1.0:1715207120.660502:0:21028:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2601 00000100:00100000:1.0:1715207120.660508:0:21028:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88006ebd1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3684:x1798523510448960:12345-0@lo:400:kworker.0 00000100:00080000:2.0:1715207120.660517:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 older than 1715207090 00000100:00080000:2.0:1715207120.660521:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 older than 1715207090 00000100:00100000:0.0:1715207120.660525:0:3686:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1715207120.660532:0:9385:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510448896 00000100:00100000:3.0:1715207120.660540:0:7122:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009e140e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3684:x1798523510448704:12345-0@lo:400:kworker.0 Request processed in 40us (225us total) trans 0 rc 0/0 00000100:00100000:2.0:1715207120.660544:0:9385:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510448896 00000100:00100000:3.0:1715207120.660551:0:7122:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2600 00000100:00100000:1.0:1715207120.660556:0:21028:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88006ebd1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-OST0001_UUID+5:3684:x1798523510448960:12345-0@lo:400:kworker.0 Request processed in 47us (213us total) trans 0 rc 0/0 00000100:00100000:0.0:1715207120.660556:0:3686:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88012f5c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3686:1798523510448832:0@lo:400:kworker.0 00000100:00100000:3.0:1715207120.660564:0:7122:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510448576 00000100:00100000:1.0:1715207120.660566:0:21028:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2601 00000100:00100000:2.0:1715207120.660567:0:9385:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1398 00000100:00100000:3.0:1715207120.660568:0:7122:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510448576 00000100:00100000:2.0:1715207120.660572:0:9385:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88012c8e1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3683:x1798523510448896:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1715207120.660585:0:21028:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2602 00000100:00100000:0.0:1715207120.660586:0:3686:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1715207120.660592:0:21028:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88012c8e2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3686:x1798523510448576:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1715207120.660598:0:9385:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88012c8e1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3683:x1798523510448896:12345-0@lo:400:kworker.0 Request processed in 26us (238us total) trans 0 rc 0/0 00000100:00100000:0.0:1715207120.660607:0:3686:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88009e369180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3686:1798523510449088:0@lo:400:kworker.0 00000100:00100000:3.0:1715207120.660613:0:3685:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1715207120.660615:0:9385:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1398 00000100:00100000:2.0:1715207120.660627:0:9385:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510448832 00000100:00100000:2.0:1715207120.660630:0:9385:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510448832 00000100:00100000:1.0:1715207120.660634:0:21028:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88012c8e2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_005:lustre-MDT0000-lwp-MDT0001_UUID+5:3686:x1798523510448576:12345-0@lo:400:kworker.0 Request processed in 42us (109us total) trans 0 rc 0/0 00000100:00100000:3.0:1715207120.660640:0:3685:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff880139bf3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3685:1798523510448768:0@lo:400:kworker.0 00000100:00100000:0.0:1715207120.660643:0:3686:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1715207120.660645:0:9385:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1399 00000100:00100000:1.0:1715207120.660645:0:21028:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2602 00000100:00100000:2.0:1715207120.660650:0:9385:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88012c8e3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3686:x1798523510448832:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1715207120.660668:0:7128:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510448512 00000100:00100000:1.0:1715207120.660673:0:7128:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510448512 00000100:00100000:3.0:1715207120.660676:0:3685:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1715207120.660680:0:9385:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88012c8e3480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3686:x1798523510448832:12345-0@lo:400:kworker.0 Request processed in 29us (96us total) trans 0 rc 0/0 00000100:00100000:2.0:1715207120.660686:0:9385:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1399 00000100:00100000:0.0:1715207120.660687:0:3686:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a0390a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3686:1798523510448576:0@lo:400:kworker.0 00000100:00100000:2.0:1715207120.660694:0:9385:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510449088 00000100:00100000:3.0:1715207120.660695:0:3685:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88012d608700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3685:1798523510449024:0@lo:400:kworker.0 00000100:00100000:2.0:1715207120.660696:0:9385:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510449088 00000100:00100000:1.0:1715207120.660698:0:7128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1172 00000100:00100000:1.0:1715207120.660705:0:7128:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88012ef09f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3685:x1798523510448512:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1715207120.660707:0:3686:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88012f5c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3686:1798523510448832:0@lo:400:kworker.0 00000100:00100000:2.0:1715207120.660711:0:9385:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1400 00000100:00100000:2.0:1715207120.660715:0:9385:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff8800adc1d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3686:x1798523510449088:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1715207120.660720:0:3685:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1715207120.660745:0:7128:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88012ef09f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3685:x1798523510448512:12345-0@lo:400:kworker.0 Request processed in 38us (132us total) trans 0 rc 0/0 00000100:00100000:0.0:1715207120.660749:0:9407:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510448768 00000100:00100000:1.0:1715207120.660753:0:7128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1172 00000100:00100000:0.0:1715207120.660755:0:9407:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510448768 00000100:00100000:3.0:1715207120.660756:0:3685:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a0390000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3685:1798523510448512:0@lo:400:kworker.0 00000100:00100000:2.0:1715207120.660761:0:9385:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff8800adc1d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3686:x1798523510449088:12345-0@lo:400:kworker.0 Request processed in 45us (119us total) trans 0 rc 0/0 00000100:00100000:2.0:1715207120.660767:0:9385:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1400 00000100:00100000:1.0:1715207120.660776:0:7124:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510449024 00000100:00100000:1.0:1715207120.660780:0:7124:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510449024 00000100:00080000:2.0:1715207120.660786:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 older than 1715207090 00000100:00080000:2.0:1715207120.660789:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 older than 1715207090 00000100:00080000:2.0:1715207120.660791:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 older than 1715207090 00000100:00100000:0.0:1715207120.660793:0:9407:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2603 00000100:00100000:2.0:1715207120.660801:0:3684:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1715207120.660802:0:9407:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009e378a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3685:x1798523510448768:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1715207120.660824:0:3684:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff880139bf2300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3684:1798523510448704:0@lo:400:kworker.0 00000100:00100000:1.0:1715207120.660826:0:7124:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2604 00000100:00100000:1.0:1715207120.660833:0:7124:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009e37ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3685:x1798523510449024:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1715207120.660834:0:3684:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88012d60ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3684:1798523510448960:0@lo:400:kworker.0 00000100:00100000:2.0:1715207120.660846:0:3684:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a0392d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3684:1798523510448448:0@lo:400:kworker.0 00000100:00100000:2.0:1715207120.660859:0:3684:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88009e0c8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3684:1798523510448384:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1715207120.660866:0:9407:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009e378a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3685:x1798523510448768:12345-0@lo:400:kworker.0 Request processed in 62us (187us total) trans 0 rc 0/0 00000100:00100000:3.0:1715207120.660871:0:9387:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510449152 00000100:00100000:3.0:1715207120.660908:0:9387:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510449152 00000100:00100000:0.0:1715207120.660912:0:9407:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2603 00000100:00100000:1.0:1715207120.660926:0:7124:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009e37ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3685:x1798523510449024:12345-0@lo:400:kworker.0 Request processed in 91us (203us total) trans 0 rc 0/0 00000100:00100000:1.0:1715207120.660936:0:7124:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2604 00000100:00100000:3.0:1715207120.660939:0:9387:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1401 00000100:00100000:3.0:1715207120.660958:0:9387:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88006ebd2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3684:x1798523510449152:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1715207120.660963:0:7118:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510448320 00000100:00100000:0.0:1715207120.660964:0:3686:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88009e369180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3686:1798523510449088:0@lo:400:kworker.0 00000100:00100000:1.0:1715207120.660967:0:7118:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510448320 00000100:00080000:2.0:1715207120.660984:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd MGS older than 1715207090 00000100:00100000:0.0:1715207120.660999:0:3683:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a0392a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3683:1798523510448640:0@lo:400:kworker.0 00000100:00100000:3.0:1715207120.661013:0:9387:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88006ebd2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3684:x1798523510449152:12345-0@lo:400:kworker.0 Request processed in 55us (209us total) trans 0 rc 0/0 00000100:00100000:0.0:1715207120.661015:0:3683:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88012f5cb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3683:1798523510448896:0@lo:400:kworker.0 00000100:00100000:3.0:1715207120.661024:0:9387:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1401 00000100:00100000:1.0:1715207120.661038:0:7118:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1067 00000100:00100000:1.0:1715207120.661046:0:7118:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff880087dff850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:24597633-7c6c-43cc-a7e8-ad821c1018c0+23:3686:x1798523510448320:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1715207120.661063:0:3685:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff880139bf3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3685:1798523510448768:0@lo:400:kworker.0 00000100:00100000:2.0:1715207120.661071:0:3684:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88012ef0a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3684:1798523510449152:0@lo:400:kworker.0 00000100:00100000:3.0:1715207120.661077:0:3685:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88012d608700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3685:1798523510449024:0@lo:400:kworker.0 00000100:00100000:1.0:1715207120.661107:0:7118:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff880087dff850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:24597633-7c6c-43cc-a7e8-ad821c1018c0+23:3686:x1798523510448320:12345-0@lo:400:kworker.0 Request processed in 61us (638us total) trans 0 rc 0/0 00000100:00100000:1.0:1715207120.661118:0:7118:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1067 00000100:00100000:0.0:1715207120.661122:0:3686:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a0392300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:24597633-7c6c-43cc-a7e8-ad821c1018c0:3686:1798523510448320:0@lo:400:kworker.0 00000100:00100000:2.0:1715207120.676067:0:3684:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88009e378e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3684:1798523510449216:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1715207120.676116:0:3684:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1715207120.676205:0:9919:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510449216 00000100:00100000:3.0:1715207120.676213:0:9919:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510449216 00000100:00100000:3.0:1715207120.676248:0:9919:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1123 00000100:00100000:3.0:1715207120.676255:0:9919:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88012ef0a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3684:x1798523510449216:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1715207120.676273:0:9919:0:(ofd_obd.c:743:ofd_statfs()) blocks cached 0 granted 16080576 pending 0 free 3921453056 avail 3689930752 00000100:00100000:3.0:1715207120.676315:0:9919:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88012ef0a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3684:x1798523510449216:12345-0@lo:13:osp-pre-1-1.0 Request processed in 60us (200us total) trans 0 rc 0/0 00000100:00100000:3.0:1715207120.676322:0:9919:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1123 00000100:00100000:2.0:1715207120.676352:0:3684:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88009e378e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3684:1798523510449216:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1715207120.678011:0:3676:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:1.0:1715207120.678156:0:7124:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523504966784 00000100:00100000:1.0:1715207120.678164:0:7124:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523504966784 00000100:00100000:1.0:1715207120.678222:0:7124:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 287 00000100:00100000:1.0:1715207120.678230:0:7124:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009e369180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+18:15535:x1798523504966784:12345-192.168.201.23@tcp:60:lhsmtool_posix.0 00000100:00080000:2.0:1715207120.678248:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 older than 1715207090 00000100:00100000:1.0:1715207120.678306:0:7124:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009e369180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+18:15535:x1798523504966784:12345-192.168.201.23@tcp:60:lhsmtool_posix.0 Request processed in 76us (302us total) trans 0 rc 0/0 00000100:00100000:1.0:1715207120.678315:0:7124:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 287 00000100:00100000:0.0:1715207120.679635:0:3678:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:1.0:1715207120.679763:0:7124:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523504966848 00000100:00100000:1.0:1715207120.679772:0:7124:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523504966848 00000100:00100000:1.0:1715207120.679828:0:7124:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 288 00000100:00100000:1.0:1715207120.679836:0:7124:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88012f5cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+7:15535:x1798523504966848:12345-192.168.201.23@tcp:60:lhsmtool_posix.0 00000100:00080000:2.0:1715207120.679851:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 older than 1715207090 00000100:00100000:1.0:1715207120.679927:0:7124:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88012f5cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+7:15535:x1798523504966848:12345-192.168.201.23@tcp:60:lhsmtool_posix.0 Request processed in 90us (298us total) trans 0 rc 0/0 00000100:00100000:1.0:1715207120.679935:0:7124:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 288 00000100:00100000:0.0:1715207120.681298:0:3677:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:3.0:1715207120.681443:0:15069:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523504966976 00000100:00100000:3.0:1715207120.681452:0:15069:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523504966976 00000100:00100000:3.0:1715207120.681500:0:15069:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 51 00000100:00100000:3.0:1715207120.681508:0:15069:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88012f5c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+18:15535:x1798523504966976:12345-192.168.201.23@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1715207120.681706:0:15069:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88012f5c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+19:15535:x1798523504966976:12345-192.168.201.23@tcp:35:lhsmtool_posix.0 Request processed in 197us (412us total) trans 4294967403 rc 0/0 00000100:00100000:3.0:1715207120.681715:0:15069:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 51 00010000:00010000:2.0:1715207121.474140:0:7113:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1715207121.940134:0:3685:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a0392300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3685:1798523510449280:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1715207121.940252:0:3685:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1715207121.940292:0:8276:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510449280 00000100:00100000:3.0:1715207121.940304:0:8276:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510449280 00000100:00100000:3.0:1715207121.940406:0:8276:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1173 00000100:00100000:3.0:1715207121.940418:0:8276:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009e0c8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3685:x1798523510449280:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1715207121.940474:0:8276:0:(mdt_handler.c:569:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1445056512 avail 1315594240 00000100:00100000:3.0:1715207121.940563:0:8276:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009e0c8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3685:x1798523510449280:12345-0@lo:41:osp-pre-0-1.0 Request processed in 146us (310us total) trans 0 rc 0/0 00000100:00100000:3.0:1715207121.940576:0:8276:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1173 00000100:00100000:2.0:1715207121.940597:0:3685:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a0392300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3685:1798523510449280:0@lo:41:osp-pre-0-1.0 00010000:00010000:2.0:1715207122.476156:0:14811:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1715207122.727010:0:15215:0:(llog.c:735:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1715207122.727018:0:15215:0:(llog_cat.c:796:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1715207122.727593:0:15215:0:(llog.c:792:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1715207122.727601:0:15215:0:(llog.c:792:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000040:00080000:0.0:1715207122.771771:0:15218:0:(llog.c:792:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 64768 count 1 00000001:02000400:1.0:1715207123.452448:0:9484:0:(debug.c:705:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12o: Layout-swap failure during Restore leaves file released ========================================================== 18:25:22 (1715207122) 00010000:00010000:3.0:1715207123.478160:0:9401:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1715207123.549820:0:3676:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:1.0:1715207123.549951:0:7124:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523504967168 00000100:00100000:1.0:1715207123.549965:0:7124:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523504967168 00000100:00080000:2.0:1715207123.550070:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 older than 1715207093 00000100:00100000:0.0:1715207123.550121:0:9407:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 289 00000100:00100000:0.0:1715207123.550135:0:9407:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff8800a0392a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+19:16375:x1798523504967168:12345-192.168.201.23@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1715207123.550173:0:9407:0:(ldlm_lockd.c:1317:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1715207123.550224:0:9407:0:(ldlm_lockd.c:1389:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fdd680/0xfa8b5bed87891d56 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7561918bd556ff31 expref: -99 pid: 9407 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1715207123.550311:0:9407:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fdcd80/0xfa8b5bed87891d5d lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 9407 initiator: MDT0 00010000:00010000:0.0:1715207123.550328:0:9407:0:(ldlm_lock.c:1089:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fdcd80/0xfa8b5bed87891d5d lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9407 initiator: MDT0 00010000:00010000:0.0:1715207123.550342:0:9407:0:(ldlm_request.c:538:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fdcd80/0xfa8b5bed87891d5d lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 9407 initiator: MDT0 00010000:00010000:0.0:1715207123.550357:0:9407:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fddb00/0xfa8b5bed87891d64 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x1:0x0].0x646966 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9407 initiator: MDT0 00010000:00010000:0.0:1715207123.550364:0:9407:0:(ldlm_lock.c:1089:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fddb00/0xfa8b5bed87891d64 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x646966 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9407 initiator: MDT0 00010000:00010000:0.0:1715207123.550381:0:9407:0:(ldlm_request.c:538:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fddb00/0xfa8b5bed87891d64 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x646966 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9407 initiator: MDT0 00000004:00200000:0.0:1715207123.550455:0:9407:0:(mdt_handler.c:938:mdt_pack_attr2body()) [0x200000002:0x2:0x0]: returning size 4096 00010000:00010000:0.0:1715207123.550527:0:9407:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fdcd80/0xfa8b5bed87891d5d lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 9407 initiator: MDT0 00010000:00010000:0.0:1715207123.550535:0:9407:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fdcd80/0xfa8b5bed87891d5d lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 9407 initiator: MDT0 00010000:00010000:0.0:1715207123.550542:0:9407:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fdcd80/0xfa8b5bed87891d5d lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9407 initiator: MDT0 00010000:00010000:0.0:1715207123.550548:0:9407:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800a8fdcd80 already unused, calling callback (ffffffffa0eef730) 00010000:00010000:0.0:1715207123.550553:0:9407:0:(ldlm_request.c:374:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fdcd80/0xfa8b5bed87891d5d lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 9407 initiator: MDT0 00010000:00010000:0.0:1715207123.550561:0:9407:0:(ldlm_request.c:1325:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fdcd80/0xfa8b5bed87891d5d lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 9407 initiator: MDT0 00010000:00010000:0.0:1715207123.550575:0:9407:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x0 (ffff8801303a6000) 00010000:00010000:0.0:1715207123.550579:0:9407:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1715207123.550583:0:9407:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fdcd80/0xfa8b5bed87891d5d lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9407 initiator: MDT0 00010000:00010000:0.0:1715207123.550589:0:9407:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fdcd80/0xfa8b5bed87891d5d lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 9407 initiator: MDT0 00010000:00010000:0.0:1715207123.550625:0:9407:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fddb00/0xfa8b5bed87891d64 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x646966 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9407 initiator: MDT0 00010000:00010000:0.0:1715207123.550634:0:9407:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fddb00/0xfa8b5bed87891d64 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x646966 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9407 initiator: MDT0 00010000:00010000:0.0:1715207123.550660:0:9407:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fddb00/0xfa8b5bed87891d64 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x646966 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9407 initiator: MDT0 00010000:00010000:0.0:1715207123.550668:0:9407:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800a8fddb00 already unused, calling callback (ffffffffa0eef730) 00010000:00010000:0.0:1715207123.550672:0:9407:0:(ldlm_request.c:374:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fddb00/0xfa8b5bed87891d64 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x646966 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9407 initiator: MDT0 00010000:00010000:0.0:1715207123.550683:0:9407:0:(ldlm_request.c:1325:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fddb00/0xfa8b5bed87891d64 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x646966 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9407 initiator: MDT0 00010000:00010000:0.0:1715207123.550697:0:9407:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x646966 (ffff8801300b4200) 00010000:00010000:0.0:1715207123.550702:0:9407:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1715207123.550706:0:9407:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fddb00/0xfa8b5bed87891d64 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x646966 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9407 initiator: MDT0 00010000:00010000:0.0:1715207123.550716:0:9407:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fddb00/0xfa8b5bed87891d64 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x646966 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9407 initiator: MDT0 00010000:00010000:0.0:1715207123.550768:0:9407:0:(ldlm_lockd.c:1559:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fdd680/0xfa8b5bed87891d56 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.23@tcp remote: 0x7561918bd556ff31 expref: 21 pid: 9407 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1715207123.550777:0:9407:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fdd680/0xfa8b5bed87891d56 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.23@tcp remote: 0x7561918bd556ff31 expref: 21 pid: 9407 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1715207123.550787:0:9407:0:(ldlm_lockd.c:1637:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8800a8fdd680, rc 0) 00000100:00100000:0.0:1715207123.550862:0:9407:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff8800a0392a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+20:16375:x1798523504967168:12345-192.168.201.23@tcp:101:lhsmtool_posix.0 Request processed in 727us (1049us total) trans 4294967404 rc 301/301 00000100:00100000:0.0:1715207123.550875:0:9407:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 289 00000100:00100000:0.0:1715207123.552463:0:3678:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:2.0:1715207123.552546:0:9407:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523504967232 00000100:00100000:2.0:1715207123.552555:0:9407:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523504967232 00000100:00100000:2.0:1715207123.552601:0:9407:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 290 00000100:00100000:2.0:1715207123.552609:0:9407:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff8800a0392d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+20:16375:x1798523504967232:12345-192.168.201.23@tcp:59:lhsmtool_posix.0 00000100:00100000:2.0:1715207123.552684:0:9407:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff8800a0392d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+20:16375:x1798523504967232:12345-192.168.201.23@tcp:59:lhsmtool_posix.0 Request processed in 74us (225us total) trans 0 rc 0/0 00000100:00100000:2.0:1715207123.552692:0:9407:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 290 00000100:00100000:0.0:1715207123.553966:0:3677:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:2.0:1715207123.554110:0:9407:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523504967296 00000100:00100000:2.0:1715207123.554118:0:9407:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523504967296 00000100:00100000:2.0:1715207123.554176:0:9407:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 291 00000100:00080000:0.0:1715207123.554180:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 older than 1715207093 00000100:00100000:2.0:1715207123.554183:0:9407:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff8800a0390000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+7:16375:x1798523504967296:12345-192.168.201.23@tcp:59:lhsmtool_posix.0 00000100:00100000:2.0:1715207123.554242:0:9407:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff8800a0390000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+7:16375:x1798523504967296:12345-192.168.201.23@tcp:59:lhsmtool_posix.0 Request processed in 58us (279us total) trans 0 rc 0/0 00000100:00100000:2.0:1715207123.554263:0:9407:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 291 00000100:00100000:0.0:1715207123.565992:0:3676:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:2.0:1715207123.566102:0:9407:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523504967616 00000100:00100000:2.0:1715207123.566112:0:9407:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523504967616 00000100:00100000:2.0:1715207123.566168:0:9407:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 292 00000100:00100000:2.0:1715207123.566179:0:9407:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff8800a0390a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+20:16378:x1798523504967616:12345-192.168.201.23@tcp:101:rm.0 00010000:00010000:2.0:1715207123.566195:0:9407:0:(ldlm_lockd.c:1317:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1715207123.566251:0:9407:0:(ldlm_lockd.c:1389:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff880072197600/0xfa8b5bed87891d72 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7561918bd556ff54 expref: -99 pid: 9407 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1715207123.566312:0:9407:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff880072197cc0/0xfa8b5bed87891d79 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 9407 initiator: MDT0 00010000:00010000:2.0:1715207123.566327:0:9407:0:(ldlm_lock.c:1089:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff880072197cc0/0xfa8b5bed87891d79 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9407 initiator: MDT0 00010000:00010000:2.0:1715207123.566340:0:9407:0:(ldlm_request.c:538:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff880072197cc0/0xfa8b5bed87891d79 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 9407 initiator: MDT0 00010000:00010000:2.0:1715207123.566357:0:9407:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff880072196880/0xfa8b5bed87891d80 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf817b0f7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 9407 initiator: MDT0 00010000:00010000:2.0:1715207123.566367:0:9407:0:(ldlm_lock.c:1089:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff880072196880/0xfa8b5bed87891d80 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf817b0f7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9407 initiator: MDT0 00010000:00010000:2.0:1715207123.566377:0:9407:0:(ldlm_request.c:538:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff880072196880/0xfa8b5bed87891d80 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf817b0f7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 9407 initiator: MDT0 00010000:00010000:2.0:1715207123.566411:0:9407:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff880072197cc0/0xfa8b5bed87891d79 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 9407 initiator: MDT0 00010000:00010000:2.0:1715207123.566420:0:9407:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff880072197cc0/0xfa8b5bed87891d79 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 9407 initiator: MDT0 00010000:00010000:2.0:1715207123.566438:0:9407:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff880072197cc0/0xfa8b5bed87891d79 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9407 initiator: MDT0 00010000:00010000:2.0:1715207123.566446:0:9407:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff880072197cc0 already unused, calling callback (ffffffffa0eef730) 00010000:00010000:2.0:1715207123.566451:0:9407:0:(ldlm_request.c:374:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff880072197cc0/0xfa8b5bed87891d79 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 9407 initiator: MDT0 00010000:00010000:2.0:1715207123.566461:0:9407:0:(ldlm_request.c:1325:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff880072197cc0/0xfa8b5bed87891d79 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 9407 initiator: MDT0 00010000:00010000:2.0:1715207123.566475:0:9407:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff88012e8ee500) 00010000:00010000:2.0:1715207123.566479:0:9407:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1715207123.566485:0:9407:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff880072197cc0/0xfa8b5bed87891d79 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9407 initiator: MDT0 00010000:00010000:2.0:1715207123.566493:0:9407:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff880072197cc0/0xfa8b5bed87891d79 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 9407 initiator: MDT0 00010000:00010000:2.0:1715207123.566532:0:9407:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff880072196880/0xfa8b5bed87891d80 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf817b0f7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 9407 initiator: MDT0 00010000:00010000:2.0:1715207123.566542:0:9407:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff880072196880/0xfa8b5bed87891d80 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf817b0f7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 9407 initiator: MDT0 00010000:00010000:2.0:1715207123.566551:0:9407:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff880072196880/0xfa8b5bed87891d80 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf817b0f7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9407 initiator: MDT0 00010000:00010000:2.0:1715207123.566558:0:9407:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff880072196880 already unused, calling callback (ffffffffa0eef730) 00010000:00010000:2.0:1715207123.566563:0:9407:0:(ldlm_request.c:374:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff880072196880/0xfa8b5bed87891d80 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf817b0f7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 9407 initiator: MDT0 00010000:00010000:2.0:1715207123.566572:0:9407:0:(ldlm_request.c:1325:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff880072196880/0xfa8b5bed87891d80 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xf817b0f7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 9407 initiator: MDT0 00010000:00010000:2.0:1715207123.566599:0:9407:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xf817b0f7 (ffff88012f833600) 00010000:00010000:2.0:1715207123.566603:0:9407:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1715207123.566608:0:9407:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff880072196880/0xfa8b5bed87891d80 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf817b0f7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9407 initiator: MDT0 00010000:00010000:2.0:1715207123.566616:0:9407:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff880072196880/0xfa8b5bed87891d80 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xf817b0f7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 9407 initiator: MDT0 00010000:00010000:2.0:1715207123.566660:0:9407:0:(ldlm_lockd.c:1559:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff880072197600/0xfa8b5bed87891d72 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.23@tcp remote: 0x7561918bd556ff54 expref: 21 pid: 9407 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1715207123.566673:0:9407:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff880072197600/0xfa8b5bed87891d72 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.23@tcp remote: 0x7561918bd556ff54 expref: 21 pid: 9407 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1715207123.566685:0:9407:0:(ldlm_lockd.c:1637:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff880072197600, rc 0) 00000100:00100000:2.0:1715207123.566748:0:9407:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff8800a0390a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+20:16378:x1798523504967616:12345-192.168.201.23@tcp:101:rm.0 Request processed in 568us (757us total) trans 0 rc 301/301 00000100:00100000:2.0:1715207123.566760:0:9407:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 292 00000040:00080000:0.0:1715207123.727134:0:15215:0:(llog.c:735:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1715207123.727144:0:15215:0:(llog_cat.c:796:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1715207123.727697:0:15215:0:(llog.c:792:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1715207123.727708:0:15215:0:(llog.c:792:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000040:00080000:0.0:1715207123.771585:0:15218:0:(llog.c:792:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 64768 count 1 00000100:00100000:0.0:1715207123.786197:0:3678:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:0.0:1715207123.786243:0:3678:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:0.0:1715207123.786291:0:3678:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:2.0:1715207123.786320:0:9407:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523504967744 00000100:00100000:1.0:1715207123.786326:0:7118:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523504967680 00000100:00100000:2.0:1715207123.786335:0:9407:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523504967744 00000100:00100000:1.0:1715207123.786338:0:7118:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523504967680 00000100:00080000:0.0:1715207123.786365:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd MGS older than 1715207093 00000100:00080000:0.0:1715207123.786370:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 older than 1715207093 00000100:00100000:2.0:1715207123.786393:0:9407:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 2605 00000100:00100000:2.0:1715207123.786403:0:9407:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff8800a0392300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:04532c2b-e5e9-420d-abb0-4480aab1e378+8:2000:x1798523504967744:12345-192.168.201.23@tcp:400:kworker.0 00000100:00100000:1.0:1715207123.786404:0:7118:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 1068 00000100:00100000:1.0:1715207123.786416:0:7118:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff880087dff450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:7e7a1b67-d841-4c3c-9f35-ded8f965b8dd+12:2001:x1798523504967680:12345-192.168.201.23@tcp:400:kworker.0 00000100:00100000:3.0:1715207123.786429:0:9387:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523504967808 00000100:00100000:3.0:1715207123.786439:0:9387:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523504967808 00000100:00100000:2.0:1715207123.786481:0:9407:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff8800a0392300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:04532c2b-e5e9-420d-abb0-4480aab1e378+8:2000:x1798523504967744:12345-192.168.201.23@tcp:400:kworker.0 Request processed in 79us (236us total) trans 0 rc 0/0 00000100:00100000:2.0:1715207123.786489:0:9407:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 2605 00000100:00080000:0.0:1715207123.786494:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 older than 1715207093 00000100:00100000:1.0:1715207123.786496:0:7118:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff880087dff450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:7e7a1b67-d841-4c3c-9f35-ded8f965b8dd+12:2001:x1798523504967680:12345-192.168.201.23@tcp:400:kworker.0 Request processed in 79us (302us total) trans 0 rc 0/0 00000100:00100000:3.0:1715207123.786497:0:9387:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 1402 00000100:00100000:3.0:1715207123.786505:0:9387:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88012f5df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+7:2001:x1798523504967808:12345-192.168.201.23@tcp:400:kworker.0 00000100:00100000:1.0:1715207123.786509:0:7118:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 1068 00000100:00100000:3.0:1715207123.786554:0:9387:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88012f5df480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+7:2001:x1798523504967808:12345-192.168.201.23@tcp:400:kworker.0 Request processed in 49us (262us total) trans 0 rc 0/0 00000100:00100000:3.0:1715207123.786562:0:9387:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 1402 00000100:00100000:1.0:1715207123.812046:0:3686:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88012d608700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3686:1798523510449344:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1715207123.812066:0:3683:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88012f5dce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3683:1798523510449408:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1715207123.812115:0:3686:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1715207123.812167:0:9919:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510449344 00000100:00100000:0.0:1715207123.812171:0:3683:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1715207123.812175:0:9919:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510449344 00000100:00100000:3.0:1715207123.812202:0:8276:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510449408 00000100:00100000:3.0:1715207123.812212:0:8276:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510449408 00000100:00100000:2.0:1715207123.812212:0:9919:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1124 00000100:00080000:0.0:1715207123.812221:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 older than 1715207093 00000100:00100000:2.0:1715207123.812230:0:9919:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88012c8d9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3686:x1798523510449344:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1715207123.812248:0:8276:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1174 00000100:00100000:3.0:1715207123.812256:0:8276:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff8801300b0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3683:x1798523510449408:12345-0@lo:41:osp-pre-1-0.0 00002000:00000024:2.0:1715207123.812262:0:9919:0:(ofd_obd.c:743:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3922501632 avail 3690979328 00000004:00000024:3.0:1715207123.812289:0:8276:0:(mdt_handler.c:569:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446313984 avail 1316851712 00000100:00100000:2.0:1715207123.812331:0:9919:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88012c8d9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3686:x1798523510449344:12345-0@lo:13:osp-pre-0-0.0 Request processed in 102us (216us total) trans 0 rc 0/0 00000100:00100000:2.0:1715207123.812340:0:9919:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1124 00000100:00100000:3.0:1715207123.812343:0:8276:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff8801300b0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3683:x1798523510449408:12345-0@lo:41:osp-pre-1-0.0 Request processed in 87us (171us total) trans 0 rc 0/0 00000100:00100000:3.0:1715207123.812351:0:8276:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1174 00000100:00100000:0.0:1715207123.812401:0:3686:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88012d608700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3686:1798523510449344:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1715207123.812454:0:3683:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88012f5dce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3683:1798523510449408:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1715207123.828029:0:3684:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88012f5dce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3684:1798523510449472:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1715207123.828093:0:3684:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1715207123.828143:0:9919:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510449472 00000100:00100000:2.0:1715207123.828149:0:9919:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510449472 00000100:00100000:2.0:1715207123.828193:0:9919:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1125 00000100:00100000:2.0:1715207123.828199:0:9919:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88012d60ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3684:x1798523510449472:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1715207123.828216:0:9919:0:(ofd_obd.c:743:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3922501632 avail 3690979328 00000100:00080000:0.0:1715207123.828222:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 older than 1715207093 00000100:00100000:2.0:1715207123.828251:0:9919:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88012d60ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3684:x1798523510449472:12345-0@lo:13:osp-pre-0-1.0 Request processed in 50us (179us total) trans 0 rc 0/0 00000100:00100000:2.0:1715207123.828262:0:9919:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1125 00000100:00100000:0.0:1715207123.828298:0:3684:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88012f5dce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3684:1798523510449472:0@lo:13:osp-pre-0-1.0 00000001:02000400:1.0:1715207124.087362:0:9549:0:(debug.c:705:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity-hsm test_12o: @@@@@@ FAIL: cannot create /mnt/lustre2/d12o.sanity-hsm/f12o.sanity-hsm with cp, status=1 00000001:00080000:0.0:1715207124.454907:0:7105:0:(tgt_lastrcvd.c:969:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967404 is committed 00010000:00010000:3.0:1715207124.480121:0:7112:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 345 lines, 345 kept, 0 dropped, 0 bad.