00010000:00010000:3.0F:1715207106.442122:0:7113:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0F:1715207106.534573:0:3677:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:3.0:1715207106.534681:0:7124:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523504964160 00000100:00100000:3.0:1715207106.534696:0:7124:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523504964160 00000100:00080000:2.0:1715207106.534757:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 older than 1715207076 00000100:00100000:3.0:1715207106.534777:0:7124:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 276 00000100:00100000:3.0:1715207106.534789:0:7124:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009e304000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+17:11187:x1798523504964160:12345-192.168.201.23@tcp:60:lhsmtool_posix.0 00000004:00080000:3.0:1715207106.535153:0:7124:0:(mdt_identity.c:116:mdt_identity_do_upcall()) lustre-MDT0000: invoked upcall /home/green/git/lustre-release/lustre/utils/l_getidentity lustre-MDT0000 0, time 310us 00000100:00100000:2.0:1715207106.539030:0:7124:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009e304000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+17:11187:x1798523504964160:12345-192.168.201.23@tcp:60:lhsmtool_posix.0 Request processed in 4237us (4460us total) trans 0 rc 0/0 00000100:00100000:2.0:1715207106.539045:0:7124:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 276 00000100:00100000:0.0F:1715207106.540201:0:3678:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:2.0:1715207106.540286:0:7124:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523504964224 00000100:00100000:2.0:1715207106.540294:0:7124:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523504964224 00000100:00100000:2.0:1715207106.540333:0:7124:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 277 00000100:00100000:2.0:1715207106.540343:0:7124:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88012d0af100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+7:11187:x1798523504964224:12345-192.168.201.23@tcp:60:lhsmtool_posix.0 00000100:00080000:2.0:1715207106.540420:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 older than 1715207076 00000004:00080000:2.0:1715207106.540810:0:7124:0:(mdt_identity.c:116:mdt_identity_do_upcall()) lustre-MDT0001: invoked upcall /home/green/git/lustre-release/lustre/utils/l_getidentity lustre-MDT0001 0, time 412us 00000100:00100000:3.0:1715207106.546273:0:7124:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88012d0af100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+7:11187:x1798523504964224:12345-192.168.201.23@tcp:60:lhsmtool_posix.0 Request processed in 5926us (6072us total) trans 0 rc 0/0 00000100:00100000:3.0:1715207106.546286:0:7124:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 277 00000100:00100000:0.0:1715207106.547484:0:3676:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:2.0:1715207106.547562:0:15069:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523504964352 00000100:00100000:2.0:1715207106.547569:0:15069:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523504964352 00000100:00100000:2.0:1715207106.547624:0:15069:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 49 00000100:00100000:2.0:1715207106.547632:0:15069:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88012d0afb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+17:11187:x1798523504964352:12345-192.168.201.23@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1715207106.547913:0:15069:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88012d0afb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+18:11187:x1798523504964352:12345-192.168.201.23@tcp:35:lhsmtool_posix.0 Request processed in 277us (427us total) trans 4294967399 rc 0/0 00000100:00100000:2.0:1715207106.547925:0:15069:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 49 00000100:00100000:3.0:1715207106.916117:0:3683:0:(ptlrpcd.c:417:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1715207106.916145:0:3683:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88009de4ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3683:1798523510445632:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1715207106.916219:0:3683:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1715207106.916297:0:7128:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510445632 00000100:00100000:2.0:1715207106.916310:0:7128:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510445632 00000100:00100000:2.0:1715207106.916398:0:7128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1161 00000100:00100000:2.0:1715207106.916408:0:7128:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009e0a9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3683:x1798523510445632:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1715207106.916449:0:7128:0:(mdt_handler.c:569:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1445056512 avail 1315594240 00000100:00100000:2.0:1715207106.916550:0:7128:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009e0a9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3683:x1798523510445632:12345-0@lo:41:osp-pre-0-1.0 Request processed in 143us (330us total) trans 0 rc 0/0 00000100:00100000:2.0:1715207106.916559:0:7128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1161 00000100:00100000:3.0:1715207106.916637:0:3683:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88009de4ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3683:1798523510445632:0@lo:41:osp-pre-0-1.0 00010000:00010000:3.0:1715207107.444101:0:14811:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1715207108.446136:0:9401:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1715207108.761994:0:3677:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:2.0:1715207108.762026:0:3677:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:2.0:1715207108.762039:0:3677:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:2.0:1715207108.762049:0:3677:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:2.0:1715207108.762067:0:3677:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:2.0:1715207108.762084:0:7119:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523504964416 00000100:00100000:2.0:1715207108.762092:0:7119:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523504964416 00000100:00100000:2.0:1715207108.762125:0:7119:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 1062 00000100:00100000:2.0:1715207108.762131:0:7119:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff8800a78d0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:7e7a1b67-d841-4c3c-9f35-ded8f965b8dd+12:2000:x1798523504964416:12345-192.168.201.23@tcp:400:kworker.0 00000100:00100000:1.0F:1715207108.762144:0:9918:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523504964672 00000100:00100000:1.0:1715207108.762153:0:9918:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523504964672 00000100:00100000:1.0:1715207108.762176:0:9918:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 1387 00000100:00100000:2.0:1715207108.762177:0:7119:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff8800a78d0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:7e7a1b67-d841-4c3c-9f35-ded8f965b8dd+12:2000:x1798523504964416:12345-192.168.201.23@tcp:400:kworker.0 Request processed in 46us (188us total) trans 0 rc 0/0 00000100:00100000:1.0:1715207108.762181:0:9918:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009e23ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+7:2000:x1798523504964672:12345-192.168.201.23@tcp:400:kworker.0 00000100:00100000:2.0:1715207108.762182:0:7119:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 1062 00000100:00100000:3.0:1715207108.762196:0:7124:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523504964480 00000100:00100000:3.0:1715207108.762203:0:7124:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523504964480 00000100:00100000:2.0:1715207108.762204:0:7123:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523504964608 00000100:00100000:2.0:1715207108.762206:0:7123:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523504964608 00000100:00100000:1.0:1715207108.762207:0:9918:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009e23ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+7:2000:x1798523504964672:12345-192.168.201.23@tcp:400:kworker.0 Request processed in 27us (140us total) trans 0 rc 0/0 00000100:00100000:1.0:1715207108.762212:0:9918:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 1387 00000100:00100000:2.0:1715207108.762221:0:7123:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523504964544 00000100:00100000:2.0:1715207108.762222:0:7123:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523504964544 00000100:00080000:1.0:1715207108.762223:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 older than 1715207078 00000100:00080000:1.0:1715207108.762226:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 older than 1715207078 00000100:00080000:1.0:1715207108.762227:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 older than 1715207078 00000100:00080000:1.0:1715207108.762228:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd MGS older than 1715207078 00000100:00100000:3.0:1715207108.762233:0:7124:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 2578 00000100:00100000:2.0:1715207108.762237:0:7123:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 2579 00000100:00100000:3.0:1715207108.762238:0:7124:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009dd8b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+7:2001:x1798523504964608:12345-192.168.201.23@tcp:400:kworker.0 00000100:00100000:2.0:1715207108.762241:0:7123:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009de4ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:04532c2b-e5e9-420d-abb0-4480aab1e378+8:2001:x1798523504964480:12345-192.168.201.23@tcp:400:kworker.0 00000100:00100000:1.0:1715207108.762242:0:6554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 2580 00000100:00100000:1.0:1715207108.762245:0:6554:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009dd8a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+18:2000:x1798523504964544:12345-192.168.201.23@tcp:400:kworker.0 00000100:00100000:2.0:1715207108.762260:0:7123:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009de4ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:04532c2b-e5e9-420d-abb0-4480aab1e378+8:2001:x1798523504964480:12345-192.168.201.23@tcp:400:kworker.0 Request processed in 19us (233us total) trans 0 rc 0/0 00000100:00100000:2.0:1715207108.762264:0:7123:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 2579 00000100:00100000:3.0:1715207108.762267:0:7124:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009dd8b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+7:2001:x1798523504964608:12345-192.168.201.23@tcp:400:kworker.0 Request processed in 29us (226us total) trans 0 rc 0/0 00000100:00100000:3.0:1715207108.762272:0:7124:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 2578 00000100:00100000:1.0:1715207108.762273:0:6554:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009dd8a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+18:2000:x1798523504964544:12345-192.168.201.23@tcp:400:kworker.0 Request processed in 27us (223us total) trans 0 rc 0/0 00000100:00100000:1.0:1715207108.762278:0:6554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 2580 00000100:00100000:3.0:1715207108.787992:0:3686:0:(ptlrpcd.c:417:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1715207108.788012:0:3686:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88009e10a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3686:1798523510445696:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1715207108.788070:0:3686:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1715207108.788110:0:7128:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510445696 00000100:00100000:2.0:1715207108.788121:0:7128:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510445696 00000100:00100000:2.0:1715207108.788174:0:7128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1162 00000100:00100000:2.0:1715207108.788185:0:7128:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009e10b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3686:x1798523510445696:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1715207108.788223:0:7128:0:(mdt_handler.c:569:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446313984 avail 1316851712 00000100:00100000:3.0:1715207108.788251:0:3686:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88006ebd1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3686:1798523510445760:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1715207108.788269:0:7128:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009e10b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3686:x1798523510445696:12345-0@lo:41:osp-pre-1-0.0 Request processed in 85us (199us total) trans 0 rc 0/0 00000100:00100000:3.0:1715207108.788285:0:3686:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1715207108.788285:0:7128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1162 00000100:00100000:3.0:1715207108.788316:0:3686:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88009e10a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3686:1798523510445696:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1715207108.788346:0:9389:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510445760 00000100:00100000:2.0:1715207108.788349:0:9389:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510445760 00000100:00080000:0.0:1715207108.788369:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 older than 1715207078 00000100:00100000:2.0:1715207108.788374:0:9389:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1112 00000100:00100000:2.0:1715207108.788378:0:9389:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88006ebd3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3686:x1798523510445760:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1715207108.788397:0:9389:0:(ofd_obd.c:743:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3922501632 avail 3690979328 00000100:00100000:2.0:1715207108.788435:0:9389:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88006ebd3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3686:x1798523510445760:12345-0@lo:13:osp-pre-0-0.0 Request processed in 57us (150us total) trans 0 rc 0/0 00000100:00100000:2.0:1715207108.788440:0:9389:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1112 00000100:00100000:3.0:1715207108.788480:0:3686:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88006ebd1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3686:1798523510445760:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1715207108.804020:0:3683:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88006ebd1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3683:1798523510445824:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1715207108.804084:0:3683:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1715207108.804154:0:9389:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510445824 00000100:00100000:2.0:1715207108.804162:0:9389:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510445824 00000100:00100000:2.0:1715207108.804204:0:9389:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1113 00000100:00100000:2.0:1715207108.804210:0:9389:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009e03d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3683:x1798523510445824:12345-0@lo:13:osp-pre-0-1.0 00000100:00080000:0.0:1715207108.804221:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 older than 1715207078 00002000:00000024:2.0:1715207108.804226:0:9389:0:(ofd_obd.c:743:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3922501632 avail 3690979328 00000100:00100000:2.0:1715207108.804259:0:9389:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009e03d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3683:x1798523510445824:12345-0@lo:13:osp-pre-0-1.0 Request processed in 49us (175us total) trans 0 rc 0/0 00000100:00100000:2.0:1715207108.804265:0:9389:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1113 00000100:00100000:0.0:1715207108.804280:0:3683:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88006ebd1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3683:1798523510445824:0@lo:13:osp-pre-0-1.0 00000001:02000400:3.0:1715207109.332714:0:7235:0:(debug.c:705:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12h: Restore a released file implicitly from a second node ========================================================== 18:25:08 (1715207108) 00010000:00010000:2.0:1715207109.448147:0:7112:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1715207109.870194:0:7300:0:(debug.c:705:libcfs_debug_mark_buffer()) DEBUG MARKER: SKIP: sanity-hsm test_12h Need 2 or more clients, have 1 00010000:00010000:2.0:1715207110.450260:0:11642:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1715207110.644036:0:3684:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88006ebd3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:24597633-7c6c-43cc-a7e8-ad821c1018c0:3684:1798523510445888:0@lo:400:kworker.0 00000100:00100000:1.0:1715207110.644101:0:3684:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1715207110.644108:0:3686:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88006ebd1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3686:1798523510446016:0@lo:400:kworker.0 00000100:00100000:0.0:1715207110.644128:0:3683:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88009e10a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3683:1798523510446080:0@lo:400:kworker.0 00000100:00100000:2.0:1715207110.644131:0:3685:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88006ebd2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3685:1798523510445952:0@lo:400:kworker.0 00000100:00100000:1.0:1715207110.644133:0:3684:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88009dc6d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3684:1798523510446144:0@lo:400:kworker.0 00000100:00100000:3.0:1715207110.644151:0:3686:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1715207110.644179:0:7118:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510445888 00000100:00100000:1.0:1715207110.644187:0:7118:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510445888 00000100:00100000:3.0:1715207110.644198:0:3686:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88009dc6d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3686:1798523510446272:0@lo:400:kworker.0 00000100:00100000:2.0:1715207110.644201:0:3685:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1715207110.644223:0:3683:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1715207110.644230:0:3685:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88009dc6c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3685:1798523510446208:0@lo:400:kworker.0 00000100:00100000:1.0:1715207110.644233:0:7118:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1063 00000100:00100000:3.0:1715207110.644236:0:3684:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1715207110.644241:0:7118:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff8800a78d0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:24597633-7c6c-43cc-a7e8-ad821c1018c0+23:3684:x1798523510445888:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1715207110.644269:0:3684:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88009dc6df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3684:1798523510446400:0@lo:400:kworker.0 00000100:00100000:0.0:1715207110.644270:0:3683:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88009dc6f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3683:1798523510446336:0@lo:400:kworker.0 00000100:00100000:2.0:1715207110.644280:0:3685:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1715207110.644292:0:7118:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff8800a78d0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:24597633-7c6c-43cc-a7e8-ad821c1018c0+23:3684:x1798523510445888:12345-0@lo:400:kworker.0 Request processed in 51us (191us total) trans 0 rc 0/0 00000100:00100000:3.0:1715207110.644325:0:3684:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1715207110.644325:0:7118:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1063 00000100:00080000:0.0:1715207110.644338:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd MGS older than 1715207080 00000100:00100000:2.0:1715207110.644340:0:3685:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88009dc6ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3685:1798523510446464:0@lo:400:kworker.0 00000100:00100000:3.0:1715207110.644349:0:3684:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88009dc6e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3684:1798523510446720:0@lo:400:kworker.0 00000100:00100000:0.0:1715207110.644361:0:9407:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510445952 00000100:00100000:0.0:1715207110.644370:0:9407:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510445952 00000100:00100000:1.0:1715207110.644371:0:6554:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510446080 00000100:00100000:3.0:1715207110.644374:0:3684:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1715207110.644375:0:6554:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510446080 00000100:00100000:2.0:1715207110.644378:0:3685:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1715207110.644401:0:3684:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88006ebd3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:24597633-7c6c-43cc-a7e8-ad821c1018c0:3684:1798523510445888:0@lo:400:kworker.0 00000100:00100000:1.0:1715207110.644404:0:6554:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510446208 00000100:00100000:1.0:1715207110.644407:0:6554:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510446208 00000100:00100000:2.0:1715207110.644424:0:3685:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88009e03e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3685:1798523510446656:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1715207110.644426:0:9407:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510446464 00000100:00100000:0.0:1715207110.644429:0:9407:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510446464 00000100:00100000:3.0:1715207110.644434:0:8276:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510446016 00000100:00100000:3.0:1715207110.644438:0:8276:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510446016 00000100:00100000:1.0:1715207110.644442:0:6554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2581 00000100:00100000:1.0:1715207110.644451:0:6554:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009e03ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0001_UUID+5:3683:x1798523510446080:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1715207110.644455:0:9407:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2582 00000100:00100000:2.0:1715207110.644461:0:3685:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1715207110.644464:0:9407:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009e35ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3685:x1798523510445952:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1715207110.644468:0:8276:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510446144 00000100:00100000:3.0:1715207110.644473:0:8276:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510446144 00000100:00100000:3.0:1715207110.644492:0:8276:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1163 00000100:00100000:3.0:1715207110.644500:0:8276:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009dc6e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3686:x1798523510446016:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1715207110.644505:0:7128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1164 00000100:00100000:2.0:1715207110.644513:0:7128:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff8801247cf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3684:x1798523510446144:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1715207110.644514:0:6554:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009e03ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-MDT0001_UUID+5:3683:x1798523510446080:12345-0@lo:400:kworker.0 Request processed in 63us (289us total) trans 0 rc 0/0 00000100:00100000:1.0:1715207110.644526:0:6554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2581 00000100:00100000:0.0:1715207110.644530:0:9407:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009e35ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3685:x1798523510445952:12345-0@lo:400:kworker.0 Request processed in 67us (328us total) trans 0 rc 0/0 00000100:00100000:0.0:1715207110.644538:0:9407:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2582 00000100:00100000:1.0:1715207110.644549:0:6554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2583 00000100:00100000:1.0:1715207110.644557:0:6554:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009e35a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3685:x1798523510446208:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1715207110.644559:0:7128:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff8801247cf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3684:x1798523510446144:12345-0@lo:400:kworker.0 Request processed in 45us (320us total) trans 0 rc 0/0 00000100:00100000:0.0:1715207110.644560:0:9407:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2584 00000100:00100000:3.0:1715207110.644563:0:8276:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009dc6e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3686:x1798523510446016:12345-0@lo:400:kworker.0 Request processed in 58us (406us total) trans 0 rc 0/0 00000100:00100000:0.0:1715207110.644565:0:9407:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff8800a03f2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3685:x1798523510446464:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1715207110.644567:0:7128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1164 00000100:00100000:3.0:1715207110.644575:0:8276:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1163 00000100:00100000:2.0:1715207110.644596:0:9385:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510446400 00000100:00100000:0.0:1715207110.644597:0:9407:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff8800a03f2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3685:x1798523510446464:12345-0@lo:400:kworker.0 Request processed in 31us (220us total) trans 0 rc 0/0 00000100:00100000:2.0:1715207110.644601:0:9385:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510446400 00000100:00100000:0.0:1715207110.644604:0:9407:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2584 00000100:00100000:1.0:1715207110.644605:0:6554:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009e35a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0000-lwp-OST0000_UUID+5:3685:x1798523510446208:12345-0@lo:400:kworker.0 Request processed in 47us (325us total) trans 0 rc 0/0 00000100:00100000:1.0:1715207110.644617:0:6554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2583 00000100:00100000:2.0:1715207110.644624:0:9385:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510446720 00000100:00100000:2.0:1715207110.644627:0:9385:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510446720 00000100:00080000:0.0:1715207110.644631:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 older than 1715207080 00000100:00100000:3.0:1715207110.644634:0:3686:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1715207110.644635:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 older than 1715207080 00000100:00100000:2.0:1715207110.644643:0:9385:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1388 00000100:00100000:2.0:1715207110.644648:0:9385:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff8800a0393100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3684:x1798523510446400:12345-0@lo:400:kworker.0 00000100:00080000:0.0:1715207110.644649:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 older than 1715207080 00000100:00100000:1.0:1715207110.644650:0:9918:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1389 00000100:00100000:1.0:1715207110.644658:0:9918:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff8800a0393b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3684:x1798523510446720:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1715207110.644664:0:3686:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88009dc6c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3686:1798523510446528:0@lo:400:kworker.0 00000100:00100000:0.0:1715207110.644664:0:3683:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1715207110.644677:0:9385:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff8800a0393100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3684:x1798523510446400:12345-0@lo:400:kworker.0 Request processed in 28us (352us total) trans 0 rc 0/0 00000100:00100000:2.0:1715207110.644683:0:9385:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1388 00000100:00100000:2.0:1715207110.644694:0:9385:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510446336 00000100:00100000:0.0:1715207110.644694:0:3683:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88009dc6c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3683:1798523510446592:0@lo:400:kworker.0 00000100:00100000:2.0:1715207110.644696:0:9385:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510446336 00000100:00100000:3.0:1715207110.644704:0:3686:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1715207110.644710:0:9918:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff8800a0393b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3684:x1798523510446720:12345-0@lo:400:kworker.0 Request processed in 51us (333us total) trans 0 rc 0/0 00000100:00100000:2.0:1715207110.644718:0:9385:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1390 00000100:00100000:2.0:1715207110.644723:0:9385:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009e03f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3683:x1798523510446336:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1715207110.644728:0:9918:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1389 00000100:00100000:0.0:1715207110.644728:0:3683:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1715207110.644737:0:3686:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88006ebd1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3686:1798523510446016:0@lo:400:kworker.0 00000100:00100000:1.0:1715207110.644745:0:9918:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510446592 00000100:00100000:2.0:1715207110.644750:0:9385:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009e03f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3683:x1798523510446336:12345-0@lo:400:kworker.0 Request processed in 26us (85us total) trans 0 rc 0/0 00000100:00100000:1.0:1715207110.644750:0:9918:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510446592 00000100:00100000:2.0:1715207110.644756:0:9385:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1390 00000100:00100000:3.0:1715207110.644768:0:6554:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510446272 00000100:00100000:3.0:1715207110.644771:0:6554:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510446272 00000100:00100000:0.0:1715207110.644772:0:3683:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88009e10a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3683:1798523510446080:0@lo:400:kworker.0 00000100:00100000:1.0:1715207110.644781:0:9918:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1391 00000100:00100000:0.0:1715207110.644786:0:3683:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88009dc6f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3683:1798523510446336:0@lo:400:kworker.0 00000100:00100000:1.0:1715207110.644789:0:9918:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009e03f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3683:x1798523510446592:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1715207110.644793:0:6554:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510446528 00000100:00100000:3.0:1715207110.644795:0:6554:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510446528 00000100:00100000:2.0:1715207110.644797:0:9389:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510446656 00000100:00100000:2.0:1715207110.644800:0:9389:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510446656 00000100:00100000:3.0:1715207110.644808:0:6554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2585 00000100:00100000:3.0:1715207110.644813:0:6554:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff8800a0391c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-lwp-OST0000_UUID+5:3686:x1798523510446272:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1715207110.644818:0:9389:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1114 00000100:00100000:2.0:1715207110.644823:0:9389:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009e141f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3685:x1798523510446656:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1715207110.644833:0:7122:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2586 00000100:00100000:1.0:1715207110.644840:0:9918:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009e03f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+5:3683:x1798523510446592:12345-0@lo:400:kworker.0 Request processed in 49us (113us total) trans 0 rc 0/0 00000100:00100000:0.0:1715207110.644840:0:7122:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff8800a0391f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3686:x1798523510446528:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1715207110.644852:0:9918:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1391 00000100:00100000:3.0:1715207110.644853:0:6554:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff8800a0391c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_006:lustre-MDT0001-lwp-OST0000_UUID+5:3686:x1798523510446272:12345-0@lo:400:kworker.0 Request processed in 37us (219us total) trans 0 rc 0/0 00000100:00100000:3.0:1715207110.644860:0:6554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2585 00002000:00000024:2.0:1715207110.644863:0:9389:0:(ofd_obd.c:743:ofd_statfs()) blocks cached 0 granted 16080576 pending 0 free 3921453056 avail 3689930752 00000100:00100000:0.0:1715207110.644874:0:7122:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff8800a0391f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3686:x1798523510446528:12345-0@lo:400:kworker.0 Request processed in 34us (170us total) trans 0 rc 0/0 00000100:00100000:0.0:1715207110.644918:0:7122:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2586 00000100:00100000:3.0:1715207110.644932:0:3686:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88009dc6d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3686:1798523510446272:0@lo:400:kworker.0 00000100:00100000:2.0:1715207110.644932:0:9389:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009e141f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3685:x1798523510446656:12345-0@lo:13:osp-pre-1-0.0 Request processed in 107us (471us total) trans 0 rc 0/0 00000100:00100000:2.0:1715207110.644940:0:9389:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1114 00000100:00080000:0.0:1715207110.644940:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 older than 1715207080 00000100:00080000:0.0:1715207110.644943:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 older than 1715207080 00000100:00100000:3.0:1715207110.644945:0:3686:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88009dc6c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3686:1798523510446528:0@lo:400:kworker.0 00000100:00080000:0.0:1715207110.644946:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 older than 1715207080 00000100:00100000:0.0:1715207110.644968:0:3683:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88009dc6c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3683:1798523510446592:0@lo:400:kworker.0 00000100:00100000:3.0:1715207110.644971:0:3684:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88009dc6d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3684:1798523510446144:0@lo:400:kworker.0 00000100:00100000:2.0:1715207110.644981:0:3685:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88006ebd2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3685:1798523510445952:0@lo:400:kworker.0 00000100:00100000:3.0:1715207110.644982:0:3684:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88009dc6df80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3684:1798523510446400:0@lo:400:kworker.0 00000100:00100000:3.0:1715207110.644991:0:3684:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88009dc6e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3684:1798523510446720:0@lo:400:kworker.0 00000100:00100000:2.0:1715207110.644994:0:3685:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88009dc6c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3685:1798523510446208:0@lo:400:kworker.0 00000100:00100000:2.0:1715207110.645005:0:3685:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88009dc6ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3685:1798523510446464:0@lo:400:kworker.0 00000100:00100000:2.0:1715207110.645020:0:3685:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88009e03e680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3685:1798523510446656:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1715207110.660073:0:3685:0:(ptlrpcd.c:417:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1715207110.660087:0:3685:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff88009e140000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3685:1798523510446784:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1715207110.660124:0:3685:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1715207110.660149:0:9919:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510446784 00000100:00100000:2.0:1715207110.660154:0:9919:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510446784 00000100:00100000:2.0:1715207110.660179:0:9919:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1115 00000100:00100000:2.0:1715207110.660185:0:9919:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009e143100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3685:x1798523510446784:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1715207110.660201:0:9919:0:(ofd_obd.c:743:ofd_statfs()) blocks cached 0 granted 16080576 pending 0 free 3921453056 avail 3689930752 00000100:00100000:2.0:1715207110.660232:0:9919:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009e143100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3685:x1798523510446784:12345-0@lo:13:osp-pre-1-1.0 Request processed in 46us (108us total) trans 0 rc 0/0 00000100:00100000:2.0:1715207110.660238:0:9919:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1115 00000100:00100000:3.0:1715207110.660265:0:3685:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff88009e140000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3685:1798523510446784:0@lo:13:osp-pre-1-1.0 00010000:00010000:3.0:1715207111.452077:0:7113:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1715207111.924085:0:3683:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a0390a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3683:1798523510446848:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1715207111.924165:0:3683:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1715207111.924227:0:7127:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510446848 00000100:00100000:3.0:1715207111.924238:0:7127:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510446848 00000100:00100000:3.0:1715207111.924302:0:7127:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1165 00000100:00100000:3.0:1715207111.924310:0:7127:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009e3a4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3683:x1798523510446848:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1715207111.924363:0:7127:0:(mdt_handler.c:569:mdt_statfs()) blocks cached 0 granted 2146304 pending 0 free 1445056512 avail 1315594240 00000100:00100000:3.0:1715207111.924433:0:7127:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009e3a4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3683:x1798523510446848:12345-0@lo:41:osp-pre-0-1.0 Request processed in 123us (268us total) trans 0 rc 0/0 00000100:00100000:3.0:1715207111.924442:0:7127:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1165 00000100:00100000:1.0:1715207111.924480:0:3683:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a0390a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3683:1798523510446848:0@lo:41:osp-pre-0-1.0 00010000:00010000:2.0:1715207112.454253:0:14811:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1715207112.454471:0:7105:0:(tgt_lastrcvd.c:969:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967399 is committed 00000001:02000400:3.0:1715207112.719103:0:7752:0:(debug.c:705:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12m: Archive/release/implicit restore ========================================================== 18:25:12 (1715207112) 00000040:00080000:0.0:1715207112.726376:0:15215:0:(llog.c:735:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1715207112.726387:0:15215:0:(llog_cat.c:796:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1715207112.726976:0:15215:0:(llog.c:792:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1715207112.726987:0:15215:0:(llog.c:792:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000040:00080000:2.0:1715207112.771735:0:15218:0:(llog.c:792:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 64768 count 1 00000100:00100000:2.0:1715207112.835514:0:3677:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:0.0:1715207112.835623:0:7122:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523504964864 00000100:00100000:0.0:1715207112.835638:0:7122:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523504964864 00000100:00080000:2.0:1715207112.835718:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 older than 1715207082 00000100:00100000:0.0:1715207112.835725:0:7122:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 278 00000100:00100000:0.0:1715207112.835735:0:7122:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009e140a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+17:14265:x1798523504964864:12345-192.168.201.23@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1715207112.835762:0:7122:0:(ldlm_lockd.c:1317:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1715207112.835919:0:7122:0:(ldlm_lockd.c:1389:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fddb00/0xfa8b5bed87891d09 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7561918bd556fedd expref: -99 pid: 7122 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1715207112.836005:0:7122:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fddf80/0xfa8b5bed87891d10 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 7122 initiator: MDT0 00010000:00010000:0.0:1715207112.836023:0:7122:0:(ldlm_lock.c:1089:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fddf80/0xfa8b5bed87891d10 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7122 initiator: MDT0 00010000:00010000:0.0:1715207112.836038:0:7122:0:(ldlm_request.c:538:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fddf80/0xfa8b5bed87891d10 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 7122 initiator: MDT0 00010000:00010000:0.0:1715207112.836052:0:7122:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fdf180/0xfa8b5bed87891d17 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x1:0x0].0x646966 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7122 initiator: MDT0 00010000:00010000:0.0:1715207112.836059:0:7122:0:(ldlm_lock.c:1089:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fdf180/0xfa8b5bed87891d17 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x646966 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7122 initiator: MDT0 00010000:00010000:0.0:1715207112.836065:0:7122:0:(ldlm_request.c:538:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fdf180/0xfa8b5bed87891d17 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x646966 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7122 initiator: MDT0 00000004:00200000:0.0:1715207112.836138:0:7122:0:(mdt_handler.c:938:mdt_pack_attr2body()) [0x200000002:0x2:0x0]: returning size 4096 00010000:00010000:0.0:1715207112.836219:0:7122:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fddf80/0xfa8b5bed87891d10 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 7122 initiator: MDT0 00010000:00010000:0.0:1715207112.836226:0:7122:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fddf80/0xfa8b5bed87891d10 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 7122 initiator: MDT0 00010000:00010000:0.0:1715207112.836233:0:7122:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fddf80/0xfa8b5bed87891d10 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7122 initiator: MDT0 00010000:00010000:0.0:1715207112.836240:0:7122:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800a8fddf80 already unused, calling callback (ffffffffa0eef730) 00010000:00010000:0.0:1715207112.836244:0:7122:0:(ldlm_request.c:374:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fddf80/0xfa8b5bed87891d10 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 7122 initiator: MDT0 00010000:00010000:0.0:1715207112.836295:0:7122:0:(ldlm_request.c:1325:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fddf80/0xfa8b5bed87891d10 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 7122 initiator: MDT0 00010000:00010000:0.0:1715207112.836311:0:7122:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x0 (ffff8801303a6000) 00010000:00010000:0.0:1715207112.836314:0:7122:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1715207112.836320:0:7122:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fddf80/0xfa8b5bed87891d10 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7122 initiator: MDT0 00010000:00010000:0.0:1715207112.836326:0:7122:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fddf80/0xfa8b5bed87891d10 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 7122 initiator: MDT0 00010000:00010000:0.0:1715207112.836364:0:7122:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fdf180/0xfa8b5bed87891d17 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x646966 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7122 initiator: MDT0 00010000:00010000:0.0:1715207112.836370:0:7122:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fdf180/0xfa8b5bed87891d17 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x646966 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7122 initiator: MDT0 00010000:00010000:0.0:1715207112.836376:0:7122:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fdf180/0xfa8b5bed87891d17 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x646966 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7122 initiator: MDT0 00010000:00010000:0.0:1715207112.836382:0:7122:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff8800a8fdf180 already unused, calling callback (ffffffffa0eef730) 00010000:00010000:0.0:1715207112.836384:0:7122:0:(ldlm_request.c:374:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fdf180/0xfa8b5bed87891d17 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x646966 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7122 initiator: MDT0 00010000:00010000:0.0:1715207112.836390:0:7122:0:(ldlm_request.c:1325:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fdf180/0xfa8b5bed87891d17 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x646966 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7122 initiator: MDT0 00010000:00010000:0.0:1715207112.836397:0:7122:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x646966 (ffff8801300b4800) 00010000:00010000:0.0:1715207112.836399:0:7122:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1715207112.836402:0:7122:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fdf180/0xfa8b5bed87891d17 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x646966 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7122 initiator: MDT0 00010000:00010000:0.0:1715207112.836407:0:7122:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fdf180/0xfa8b5bed87891d17 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x646966 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7122 initiator: MDT0 00010000:00010000:0.0:1715207112.836458:0:7122:0:(ldlm_lockd.c:1559:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fddb00/0xfa8b5bed87891d09 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.23@tcp remote: 0x7561918bd556fedd expref: 19 pid: 7122 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1715207112.836578:0:7122:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff8800a8fddb00/0xfa8b5bed87891d09 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.23@tcp remote: 0x7561918bd556fedd expref: 19 pid: 7122 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1715207112.836588:0:7122:0:(ldlm_lockd.c:1637:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff8800a8fddb00, rc 0) 00000100:00100000:0.0:1715207112.836663:0:7122:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009e140a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+18:14265:x1798523504964864:12345-192.168.201.23@tcp:101:lhsmtool_posix.0 Request processed in 928us (1159us total) trans 4294967400 rc 301/301 00000100:00100000:0.0:1715207112.836673:0:7122:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 278 00000100:00100000:2.0:1715207112.839210:0:3678:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:0.0:1715207112.839301:0:7122:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523504964928 00000100:00100000:0.0:1715207112.839311:0:7122:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523504964928 00000100:00100000:0.0:1715207112.839360:0:7122:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 279 00000100:00100000:0.0:1715207112.839371:0:7122:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009e143800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+18:14265:x1798523504964928:12345-192.168.201.23@tcp:59:lhsmtool_posix.0 00000100:00100000:0.0:1715207112.839463:0:7122:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009e143800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+18:14265:x1798523504964928:12345-192.168.201.23@tcp:59:lhsmtool_posix.0 Request processed in 91us (256us total) trans 0 rc 0/0 00000100:00100000:0.0:1715207112.839475:0:7122:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 279 00000100:00100000:1.0:1715207112.840613:0:3676:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:0.0:1715207112.840684:0:7122:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523504964992 00000100:00100000:0.0:1715207112.840692:0:7122:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523504964992 00000100:00100000:0.0:1715207112.840752:0:7122:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 280 00000100:00100000:0.0:1715207112.840762:0:7122:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009e3a5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+7:14265:x1798523504964992:12345-192.168.201.23@tcp:59:lhsmtool_posix.0 00000100:00080000:2.0:1715207112.840778:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 older than 1715207082 00000100:00100000:0.0:1715207112.840829:0:7122:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009e3a5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+7:14265:x1798523504964992:12345-192.168.201.23@tcp:59:lhsmtool_posix.0 Request processed in 66us (223us total) trans 0 rc 0/0 00000100:00100000:0.0:1715207112.840840:0:7122:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 280 00000100:00100000:0.0:1715207112.850591:0:3677:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:2.0:1715207112.850701:0:7122:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523504965312 00000100:00100000:2.0:1715207112.850710:0:7122:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523504965312 00000100:00100000:2.0:1715207112.850760:0:7122:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 281 00000100:00100000:2.0:1715207112.850768:0:7122:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009e03e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+18:14268:x1798523504965312:12345-192.168.201.23@tcp:101:rm.0 00010000:00010000:2.0:1715207112.850783:0:7122:0:(ldlm_lockd.c:1317:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1715207112.850801:0:7122:0:(ldlm_lockd.c:1389:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff880072194480/0xfa8b5bed87891d25 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x7561918bd556ff00 expref: -99 pid: 7122 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1715207112.850849:0:7122:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff880072195f80/0xfa8b5bed87891d2c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 7122 initiator: MDT0 00010000:00010000:2.0:1715207112.850862:0:7122:0:(ldlm_lock.c:1089:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff880072195f80/0xfa8b5bed87891d2c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 7122 initiator: MDT0 00010000:00010000:2.0:1715207112.850873:0:7122:0:(ldlm_request.c:538:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff880072195f80/0xfa8b5bed87891d2c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 7122 initiator: MDT0 00010000:00010000:2.0:1715207112.850918:0:7122:0:(ldlm_lock.c:791:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff880072197180/0xfa8b5bed87891d33 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xe617b0f7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 7122 initiator: MDT0 00010000:00010000:2.0:1715207112.850925:0:7122:0:(ldlm_lock.c:1089:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff880072197180/0xfa8b5bed87891d33 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xe617b0f7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7122 initiator: MDT0 00010000:00010000:2.0:1715207112.850932:0:7122:0:(ldlm_request.c:538:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff880072197180/0xfa8b5bed87891d33 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xe617b0f7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 7122 initiator: MDT0 00010000:00010000:2.0:1715207112.850967:0:7122:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff880072195f80/0xfa8b5bed87891d2c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 7122 initiator: MDT0 00010000:00010000:2.0:1715207112.850973:0:7122:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff880072195f80/0xfa8b5bed87891d2c lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 7122 initiator: MDT0 00010000:00010000:2.0:1715207112.850980:0:7122:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff880072195f80/0xfa8b5bed87891d2c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 7122 initiator: MDT0 00010000:00010000:2.0:1715207112.850995:0:7122:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff880072195f80 already unused, calling callback (ffffffffa0eef730) 00010000:00010000:2.0:1715207112.850999:0:7122:0:(ldlm_request.c:374:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff880072195f80/0xfa8b5bed87891d2c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 7122 initiator: MDT0 00010000:00010000:2.0:1715207112.851006:0:7122:0:(ldlm_request.c:1325:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff880072195f80/0xfa8b5bed87891d2c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 7122 initiator: MDT0 00010000:00010000:2.0:1715207112.851018:0:7122:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff88012e8ee500) 00010000:00010000:2.0:1715207112.851021:0:7122:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1715207112.851025:0:7122:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff880072195f80/0xfa8b5bed87891d2c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 7122 initiator: MDT0 00010000:00010000:2.0:1715207112.851031:0:7122:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff880072195f80/0xfa8b5bed87891d2c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 7122 initiator: MDT0 00010000:00010000:2.0:1715207112.851061:0:7122:0:(ldlm_lock.c:845:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff880072197180/0xfa8b5bed87891d33 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xe617b0f7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 7122 initiator: MDT0 00010000:00010000:2.0:1715207112.851067:0:7122:0:(ldlm_lock.c:906:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff880072197180/0xfa8b5bed87891d33 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xe617b0f7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 7122 initiator: MDT0 00010000:00010000:2.0:1715207112.851073:0:7122:0:(ldlm_lockd.c:1947:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff880072197180/0xfa8b5bed87891d33 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xe617b0f7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7122 initiator: MDT0 00010000:00010000:2.0:1715207112.851078:0:7122:0:(ldlm_lockd.c:1961:ldlm_handle_bl_callback()) Lock ffff880072197180 already unused, calling callback (ffffffffa0eef730) 00010000:00010000:2.0:1715207112.851080:0:7122:0:(ldlm_request.c:374:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff880072197180/0xfa8b5bed87891d33 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xe617b0f7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 7122 initiator: MDT0 00010000:00010000:2.0:1715207112.851086:0:7122:0:(ldlm_request.c:1325:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff880072197180/0xfa8b5bed87891d33 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xe617b0f7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 7122 initiator: MDT0 00010000:00010000:2.0:1715207112.851092:0:7122:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xe617b0f7 (ffff88012f833200) 00010000:00010000:2.0:1715207112.851094:0:7122:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1715207112.851103:0:7122:0:(ldlm_lockd.c:1971:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff880072197180/0xfa8b5bed87891d33 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xe617b0f7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7122 initiator: MDT0 00010000:00010000:2.0:1715207112.851109:0:7122:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff880072197180/0xfa8b5bed87891d33 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xe617b0f7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 7122 initiator: MDT0 00010000:00010000:2.0:1715207112.851145:0:7122:0:(ldlm_lockd.c:1559:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff880072194480/0xfa8b5bed87891d25 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.23@tcp remote: 0x7561918bd556ff00 expref: 19 pid: 7122 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1715207112.851154:0:7122:0:(ldlm_lock.c:215:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff880072194480/0xfa8b5bed87891d25 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.201.23@tcp remote: 0x7561918bd556ff00 expref: 19 pid: 7122 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1715207112.851163:0:7122:0:(ldlm_lockd.c:1637:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff880072194480, rc 0) 00000100:00100000:2.0:1715207112.851201:0:7122:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009e03e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+18:14268:x1798523504965312:12345-192.168.201.23@tcp:101:rm.0 Request processed in 433us (615us total) trans 0 rc 301/301 00000100:00100000:2.0:1715207112.851210:0:7122:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 281 00000001:02000400:2.0:1715207113.381201:0:7817:0:(debug.c:705:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity-hsm test_12m: @@@@@@ FAIL: cannot create /mnt/lustre2/d12m.sanity-hsm/f12m.sanity-hsm with cp, status=1 00010000:00010000:3.0:1715207113.456113:0:9401:0:(ldlm_lockd.c:2860:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1715207113.727020:0:15215:0:(llog.c:735:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1715207113.727028:0:15215:0:(llog_cat.c:796:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1715207113.727746:0:15215:0:(llog.c:792:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1715207113.727762:0:15215:0:(llog.c:792:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1715207113.770009:0:3678:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:0.0:1715207113.770066:0:3678:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:0.0:1715207113.770086:0:3678:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:0.0:1715207113.770116:0:3678:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:3.0:1715207113.770136:0:7124:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523504965504 00000100:00100000:3.0:1715207113.770152:0:7124:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523504965504 00000100:00100000:0.0:1715207113.770153:0:3678:0:(events.c:359:request_in_callback()) peer: 12345-192.168.201.23@tcp (source: 12345-192.168.201.23@tcp) 00000100:00100000:2.0:1715207113.770159:0:7119:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523504965376 00000100:00100000:2.0:1715207113.770168:0:7119:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523504965376 00000100:00100000:0.0:1715207113.770177:0:10664:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523504965440 00000100:00100000:0.0:1715207113.770183:0:10664:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523504965440 00000100:00100000:3.0:1715207113.770200:0:7124:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523504965568 00000100:00100000:3.0:1715207113.770204:0:7124:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523504965568 00000100:00100000:2.0:1715207113.770216:0:7119:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 1064 00000100:00100000:0.0:1715207113.770216:0:10664:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 2587 00000100:00100000:0.0:1715207113.770226:0:10664:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009dc6ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+18:2001:x1798523504965504:12345-192.168.201.23@tcp:400:kworker.0 00000100:00100000:2.0:1715207113.770228:0:7119:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff8800a78d0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:7e7a1b67-d841-4c3c-9f35-ded8f965b8dd+12:2000:x1798523504965376:12345-192.168.201.23@tcp:400:kworker.0 00000100:00100000:3.0:1715207113.770230:0:7124:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 2588 00000100:00100000:1.0:1715207113.770230:0:11685:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523504965632 00000100:00100000:3.0:1715207113.770238:0:7124:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009dc6c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:04532c2b-e5e9-420d-abb0-4480aab1e378+8:2000:x1798523504965440:12345-192.168.201.23@tcp:400:kworker.0 00000100:00100000:1.0:1715207113.770240:0:11685:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523504965632 00000100:00100000:1.0:1715207113.770304:0:11685:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 1392 00000100:00100000:1.0:1715207113.770311:0:11685:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009dc6e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+7:2001:x1798523504965632:12345-192.168.201.23@tcp:400:kworker.0 00000100:00100000:0.0:1715207113.770321:0:10664:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009dc6ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+18:2001:x1798523504965504:12345-192.168.201.23@tcp:400:kworker.0 Request processed in 95us (252us total) trans 0 rc 0/0 00000100:00100000:2.0:1715207113.770323:0:7119:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff8800a78d0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:7e7a1b67-d841-4c3c-9f35-ded8f965b8dd+12:2000:x1798523504965376:12345-192.168.201.23@tcp:400:kworker.0 Request processed in 98us (324us total) trans 0 rc 0/0 00000100:00100000:0.0:1715207113.770330:0:10664:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 2587 00000100:00100000:2.0:1715207113.770332:0:7119:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 1064 00000100:00100000:0.0:1715207113.770350:0:10664:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.201.23@tcp, seq: 2589 00000100:00100000:0.0:1715207113.770356:0:10664:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009dc6df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+7:2000:x1798523504965568:12345-192.168.201.23@tcp:400:kworker.0 00000100:00100000:1.0:1715207113.770358:0:11685:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009dc6e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+7:2001:x1798523504965632:12345-192.168.201.23@tcp:400:kworker.0 Request processed in 46us (241us total) trans 0 rc 0/0 00000100:00100000:1.0:1715207113.770366:0:11685:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 1392 00000100:00080000:2.0:1715207113.770369:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 older than 1715207083 00000100:00080000:2.0:1715207113.770373:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0001 older than 1715207083 00000100:00080000:2.0:1715207113.770375:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd MGS older than 1715207083 00000100:00100000:0.0:1715207113.770381:0:10664:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009dc6df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:1bc50a3e-f07c-4b55-83c3-5f3c3d406528+7:2000:x1798523504965568:12345-192.168.201.23@tcp:400:kworker.0 Request processed in 24us (227us total) trans 0 rc 0/0 00000100:00100000:0.0:1715207113.770388:0:10664:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 2589 00000100:00100000:3.0:1715207113.770504:0:7124:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009dc6c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:04532c2b-e5e9-420d-abb0-4480aab1e378+8:2000:x1798523504965440:12345-192.168.201.23@tcp:400:kworker.0 Request processed in 265us (415us total) trans 0 rc 0/0 00000100:00100000:3.0:1715207113.770513:0:7124:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.201.23@tcp, seq: 2588 00000040:00080000:2.0:1715207113.771582:0:15218:0:(llog.c:792:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 64768 count 1 00000100:00100000:3.0:1715207113.796046:0:3685:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a0390000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3685:1798523510446976:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1715207113.796103:0:3685:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1715207113.796104:0:3683:0:(ptlrpcd.c:417:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1715207113.796125:0:3683:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a0391180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3683:1798523510446912:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1715207113.796144:0:9919:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510446976 00000100:00100000:2.0:1715207113.796158:0:9919:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510446976 00000100:00100000:1.0:1715207113.796170:0:3683:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1715207113.796192:0:7127:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510446912 00000100:00100000:3.0:1715207113.796198:0:7127:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510446912 00000100:00100000:2.0:1715207113.796198:0:9919:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1116 00000100:00100000:2.0:1715207113.796206:0:9919:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff8800a0390380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3685:x1798523510446976:12345-0@lo:13:osp-pre-0-0.0 00000100:00080000:1.0:1715207113.796224:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 older than 1715207083 00000100:00100000:3.0:1715207113.796227:0:7127:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1166 00000100:00100000:3.0:1715207113.796233:0:7127:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff88009e3a6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3683:x1798523510446912:12345-0@lo:41:osp-pre-1-0.0 00002000:00000024:2.0:1715207113.796235:0:9919:0:(ofd_obd.c:743:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3922501632 avail 3690979328 00000004:00000024:3.0:1715207113.796282:0:7127:0:(mdt_handler.c:569:mdt_statfs()) blocks cached 0 granted 4292608 pending 0 free 1446313984 avail 1316851712 00000100:00100000:2.0:1715207113.796312:0:9919:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff8800a0390380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3685:x1798523510446976:12345-0@lo:13:osp-pre-0-0.0 Request processed in 105us (208us total) trans 0 rc 0/0 00000100:00100000:2.0:1715207113.796321:0:9919:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1116 00000100:00100000:3.0:1715207113.796339:0:7127:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff88009e3a6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3683:x1798523510446912:12345-0@lo:41:osp-pre-1-0.0 Request processed in 103us (167us total) trans 0 rc 0/0 00000100:00100000:3.0:1715207113.796347:0:7127:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1166 00000100:00100000:3.0:1715207113.796393:0:3685:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a0390000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3685:1798523510446976:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1715207113.796394:0:3683:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a0391180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3683:1798523510446912:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1715207113.811976:0:3686:0:(client.c:1776:ptlrpc_send_new_req()) Sending RPC req@ffff8800a0390000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3686:1798523510447040:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1715207113.812018:0:3686:0:(events.c:359:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1715207113.812647:0:9919:0:(service.c:2096:ptlrpc_server_handle_req_in()) unwrap req x1798523510447040 00000100:00100000:2.0:1715207113.812655:0:9919:0:(service.c:2161:ptlrpc_server_handle_req_in()) got req x1798523510447040 00000100:00100000:2.0:1715207113.812689:0:9919:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1117 00000100:00100000:2.0:1715207113.812697:0:9919:0:(service.c:2335:ptlrpc_server_handle_request()) Handling RPC req@ffff8800a0390e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3686:x1798523510447040:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1715207113.812715:0:9919:0:(ofd_obd.c:743:ofd_statfs()) blocks cached 0 granted 278208 pending 0 free 3922501632 avail 3690979328 00000100:00100000:2.0:1715207113.812772:0:9919:0:(service.c:2384:ptlrpc_server_handle_request()) Handled RPC req@ffff8800a0390e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3686:x1798523510447040:12345-0@lo:13:osp-pre-0-1.0 Request processed in 76us (753us total) trans 0 rc 0/0 00000100:00100000:2.0:1715207113.812780:0:9919:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1117 00000100:00080000:2.0:1715207113.812798:0:7120:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 older than 1715207083 00000100:00100000:3.0:1715207113.812817:0:3686:0:(client.c:2261:ptlrpc_check_set()) Completed RPC req@ffff8800a0390000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3686:1798523510447040:0@lo:13:osp-pre-0-1.0 Debug log: 450 lines, 450 kept, 0 dropped, 0 bad.