00000100:00100000:2.0F:1769754720.918265:0:3318:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1769754720.918317:0:3318:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb98566d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3318:1855720644836864:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1769754720.918408:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769754720.918529:0:42939:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644836864 00000100:00100000:2.0:1769754720.918540:0:42939:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644836864 00000100:00100000:2.0:1769754720.918834:0:42939:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 38 00000100:00100000:2.0:1769754720.918846:0:42939:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb98567100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3318:x1855720644836864:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1769754720.918893:0:42939:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 45940736 pending 0 free 3694133248 avail 3692036096 00000100:00100000:2.0:1769754720.918945:0:42939:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb98567100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3318:x1855720644836864:12345-0@lo:13:osp-pre-0-0.0 Request processed in 101us (536us total) trans 0 rc 0/0 00000100:00100000:2.0:1769754720.918954:0:42939:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 38 00000100:00080000:2.0:1769754720.918981:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1769754720.919016:0:3318:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 34359738382 gen 2 00000004:00080000:2.0:1769754720.919024:0:3318:0:(osp_trans.c:1035:osp_request_commit_cb()) trans no 34359738381 committed transno 34359738382 00000004:00080000:2.0:1769754720.919055:0:3318:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff90cbaf978700, transno 34359738382 00000100:00100000:2.0:1769754720.919074:0:3318:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb98566d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3318:1855720644836864:0@lo:13:osp-pre-0-0.0 00010000:00010000:3.0F:1769754720.979693:0:42301:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1769754722.005727:0:42302:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769754722.102556:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:3.0:1769754722.102611:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:3.0:1769754722.102660:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:3.0:1769754722.102687:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:3.0:1769754722.102719:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:3.0:1769754722.102754:0:43451:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624208256 00000100:00100000:3.0:1769754722.102769:0:43451:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624208256 00000100:00100000:3.0:1769754722.102852:0:43451:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624208384 00000100:00100000:3.0:1769754722.102856:0:43451:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624208384 00000100:00100000:3.0:1769754722.102884:0:43451:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 65 00000100:00100000:3.0:1769754722.102896:0:43451:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb8710ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+5:2369:x1855720624208256:12345-192.168.202.6@tcp:400:kworker.0 00000100:00100000:3.0:1769754722.102987:0:43451:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb8710ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+5:2369:x1855720624208256:12345-192.168.202.6@tcp:400:kworker.0 Request processed in 93us (374us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754722.102996:0:43451:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 65 00000100:00100000:3.0:1769754722.103017:0:43451:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 66 00000100:00100000:3.0:1769754722.103023:0:43451:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb87862d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+6:2369:x1855720624208384:12345-192.168.202.6@tcp:400:kworker.0 00000100:00100000:3.0:1769754722.103049:0:43451:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb87862d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+6:2369:x1855720624208384:12345-192.168.202.6@tcp:400:kworker.0 Request processed in 27us (330us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754722.103056:0:43451:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 66 00000100:00100000:3.0:1769754722.103077:0:42315:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624208512 00000100:00100000:3.0:1769754722.103081:0:42315:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624208512 00000100:00100000:3.0:1769754722.103111:0:42315:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624208128 00000100:00100000:3.0:1769754722.103115:0:42315:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624208128 00000100:00100000:3.0:1769754722.103134:0:42315:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 84 00000100:00100000:3.0:1769754722.103141:0:42315:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb87862300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:99ce4681-381f-407c-90f1-2ef98266ced9+5:2369:x1855720624208512:12345-192.168.202.6@tcp:400:kworker.0 00000100:00100000:3.0:1769754722.103173:0:42315:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb87862300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:99ce4681-381f-407c-90f1-2ef98266ced9+5:2369:x1855720624208512:12345-192.168.202.6@tcp:400:kworker.0 Request processed in 32us (513us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754722.103187:0:42315:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 84 00000100:00100000:3.0:1769754722.103216:0:42315:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 85 00000100:00100000:3.0:1769754722.103222:0:42315:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb87861f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+13:2369:x1855720624208128:12345-192.168.202.6@tcp:400:kworker.0 00000100:00100000:3.0:1769754722.103250:0:42315:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb87861f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+13:2369:x1855720624208128:12345-192.168.202.6@tcp:400:kworker.0 Request processed in 29us (563us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754722.103257:0:42315:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 85 00000100:00080000:3.0:1769754722.103292:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1769754722.103298:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1769754722.103300:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1769754722.104136:0:42308:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624208000 00000100:00100000:2.0:1769754722.104146:0:42308:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624208000 00000100:00100000:2.0:1769754722.104207:0:42308:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 146 00000100:00100000:2.0:1769754722.104216:0:42308:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbb955b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:cd1d8275-6a1a-49a3-b443-af2f7055b864+12:2369:x1855720624208000:12345-192.168.202.6@tcp:400:kworker.0 00000100:00100000:2.0:1769754722.104279:0:42308:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbb955b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:cd1d8275-6a1a-49a3-b443-af2f7055b864+12:2369:x1855720624208000:12345-192.168.202.6@tcp:400:kworker.0 Request processed in 65us (1738us total) trans 0 rc 0/0 00000100:00100000:2.0:1769754722.104286:0:42308:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 146 00000020:00080000:0.0F:1769754722.517916:0:5807:0:(update_trans.c:1333:distribute_txn_batchid_cb()) lustre-MDT0000: 30064771787 batchid updated 00000020:00080000:2.0:1769754722.517988:0:42352:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 30064771787 00000020:00080000:2.0:1769754722.518001:0:42352:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 30064771787 00010000:00010000:2.0:1769754723.028765:0:43453:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769754723.507946:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:0.0:1769754723.508067:0:42300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624208768 00000100:00100000:0.0:1769754723.508083:0:42300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624208768 00000100:00100000:0.0:1769754723.508172:0:42300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 14 00000100:00100000:0.0:1769754723.508184:0:42300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb87860000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+13:49738:x1855720624208768:12345-192.168.202.6@tcp:103:lctl.0 00010000:00010000:0.0:1769754723.508207:0:42300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:0.0:1769754723.508242:0:42300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81128e00/0x11e65116e9df240e lrc: 0/0,0 mode: --/PR res: [0x200003ab1:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd0f expref: 13 pid: 42315 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754723.508268:0:42300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81128c00/0x11e65116e9df23dd lrc: 0/0,0 mode: --/CR res: [0x200003ab1:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd08 expref: 12 pid: 42315 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754723.508280:0:42300:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x14:0x0].0x0 (ffff90cb83c8fa00) 00010000:00010000:0.0:1769754723.508292:0:42300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1769754723.508372:0:42300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb87860000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+11:49738:x1855720624208768:12345-192.168.202.6@tcp:103:lctl.0 Request processed in 190us (441us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754723.508381:0:42300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 14 00000100:00100000:0.0:1769754723.510854:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:3.0:1769754723.510934:0:43811:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624208896 00000100:00100000:3.0:1769754723.510947:0:43811:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624208896 00000100:00100000:3.0:1769754723.511023:0:43811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 180 00000100:00100000:3.0:1769754723.511035:0:43811:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbb8704e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+11:49738:x1855720624208896:12345-192.168.202.6@tcp:58:lctl.0 00010000:00010000:3.0:1769754723.511174:0:43811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c37a00/0x11e65116e9df243f lrc: 3/0,1 mode: --/EX res: [0x200003ab1:0x14:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754723.511192:0:43811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c37a00/0x11e65116e9df243f lrc: 3/0,1 mode: EX/EX res: [0x200003ab1:0x14:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754723.511206:0:43811:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c37a00/0x11e65116e9df243f lrc: 3/0,1 mode: EX/EX res: [0x200003ab1:0x14:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1769754723.511545:0:43811:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x32:0x0].4, 136 off8736 00000100:00100000:3.0:1769754723.511661:0:43811:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbb8704e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+11:49738:x1855720624208896:12345-192.168.202.6@tcp:58:lctl.0 Request processed in 627us (817us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754723.511671:0:43811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 180 00000100:00100000:1.0F:1769754723.515386:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:3.0:1769754723.515582:0:42313:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624209024 00000100:00100000:3.0:1769754723.515596:0:42313:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624209024 00000100:00100000:3.0:1769754723.515702:0:42313:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 181 00000100:00100000:3.0:1769754723.515713:0:42313:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbaf978700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+11:49738:x1855720624209024:12345-192.168.202.6@tcp:101:lctl.0 00010000:00010000:3.0:1769754723.515741:0:42313:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769754723.515772:0:42313:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36600/0x11e65116e9df2446 lrc: 2/0,0 mode: --/CR res: [0x200003ab1:0x14:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x2b94eb038208fd1d expref: -99 pid: 42313 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754723.515864:0:42313:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c37a00/0x11e65116e9df243f lrc: 2/0,1 mode: EX/EX res: [0x200003ab1:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754723.515876:0:42313:0:(ldlm_resource.c:1714:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36600/0x11e65116e9df2446 lrc: 4/0,0 mode: --/CR res: [0x200003ab1:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd1d expref: 12 pid: 42313 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754723.515901:0:42313:0:(ldlm_request.c:362:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c37a00/0x11e65116e9df243f lrc: 3/0,1 mode: EX/EX res: [0x200003ab1:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754723.515919:0:42313:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36600/0x11e65116e9df2446 lrc: 3/0,0 mode: --/CR res: [0x200003ab1:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd1d expref: 12 pid: 42313 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754723.515937:0:42313:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x14:0x0].0x0 (ffff90cb87176500) 00010000:00010000:3.0:1769754723.515941:0:42313:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1769754723.515945:0:42313:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36600/0x11e65116e9df2446 lrc: 3/0,0 mode: --/CR res: [0x200003ab1:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd1d expref: 12 pid: 42313 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754723.515954:0:42313:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff90ca82c36600, rc 0) 00000100:00100000:3.0:1769754723.516026:0:42313:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbaf978700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+12:49738:x1855720624209024:12345-192.168.202.6@tcp:101:lctl.0 Request processed in 315us (657us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754723.516034:0:42313:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 181 00000100:00100000:2.0:1769754723.991826:0:3318:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cbb95f2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3318:1855720644837248:0@lo:400:kworker.0 00000100:00100000:2.0:1769754723.991940:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769754723.992035:0:3319:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90ca86de2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3319:1855720644837376:0@lo:400:kworker.0 00000100:00100000:2.0:1769754723.992064:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769754723.992266:0:3320:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb98566d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:12a461d6-d4b3-45fe-9a31-84fa5ea63cfd:3320:1855720644836992:0@lo:400:kworker.0 00000100:00100000:0.0:1769754723.992295:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769754723.992320:0:3320:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90ca86de2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3320:1855720644837504:0@lo:400:kworker.0 00000100:00100000:0.0:1769754723.992339:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769754723.993888:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1769754723.993905:0:3320:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cbb95f1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3320:1855720644837120:0@lo:400:kworker.0 00000100:00100000:0.0:1769754723.993953:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769754723.993974:0:3320:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90ca86de1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3320:1855720644837632:0@lo:400:kworker.0 00000100:00100000:0.0:1769754723.993997:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769754723.994050:0:43811:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644837376 00000100:00100000:0.0:1769754723.994059:0:43811:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644837376 00000100:00100000:0.0:1769754723.994101:0:43811:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644837504 00000100:00100000:0.0:1769754723.994105:0:43811:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644837504 00000100:00100000:0.0:1769754723.994124:0:43811:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644837632 00000100:00100000:0.0:1769754723.994127:0:43811:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644837632 00000100:00100000:0.0:1769754723.994142:0:43811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 86 00000100:00100000:0.0:1769754723.994150:0:43811:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90ca86de3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3319:x1855720644837376:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769754723.994208:0:43811:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90ca86de3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3319:x1855720644837376:12345-0@lo:400:kworker.0 Request processed in 59us (2142us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754723.994216:0:43811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 86 00000100:00100000:0.0:1769754723.994232:0:43811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 87 00000100:00100000:0.0:1769754723.994237:0:43811:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbb8704700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1855720644837504:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769754723.994268:0:43811:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbb8704700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1855720644837504:12345-0@lo:400:kworker.0 Request processed in 31us (1932us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754723.994275:0:43811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 87 00000100:00100000:0.0:1769754723.994285:0:43811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 88 00000100:00100000:0.0:1769754723.994291:0:43811:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbb8705880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3320:x1855720644837632:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769754723.994316:0:43811:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbb8705880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3320:x1855720644837632:12345-0@lo:400:kworker.0 Request processed in 26us (319us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754723.994322:0:43811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 88 00000100:00100000:0.0:1769754723.994362:0:3319:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90ca86de2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3319:1855720644837376:0@lo:400:kworker.0 00000100:00100000:0.0:1769754723.994391:0:3320:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90ca86de1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3320:1855720644837632:0@lo:400:kworker.0 00000100:00100000:0.0:1769754723.994401:0:3320:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90ca86de2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3320:1855720644837504:0@lo:400:kworker.0 00000100:00100000:3.0:1769754723.996049:0:42936:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644837248 00000100:00100000:3.0:1769754723.996060:0:42936:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644837248 00000100:00100000:3.0:1769754723.996120:0:42936:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644837120 00000100:00100000:3.0:1769754723.996123:0:42936:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644837120 00000100:00100000:3.0:1769754723.996147:0:42936:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 67 00000100:00100000:3.0:1769754723.996154:0:42936:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90ca88ca9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3318:x1855720644837248:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769754723.996224:0:42936:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90ca88ca9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3318:x1855720644837248:12345-0@lo:400:kworker.0 Request processed in 71us (4284us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754723.996230:0:42936:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 67 00000100:00100000:3.0:1769754723.996243:0:42936:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 68 00000100:00100000:3.0:1769754723.996248:0:42936:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbb8707100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3320:x1855720644837120:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769754723.996279:0:42936:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbb8707100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3320:x1855720644837120:12345-0@lo:400:kworker.0 Request processed in 31us (2326us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754723.996284:0:42936:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 68 00000100:00080000:3.0:1769754723.996329:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1769754723.996333:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1769754723.996335:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1769754723.996369:0:3318:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cbb95f2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3318:1855720644837248:0@lo:400:kworker.0 00000100:00100000:3.0:1769754723.996411:0:3320:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cbb95f1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3320:1855720644837120:0@lo:400:kworker.0 00000100:00100000:2.0:1769754723.999867:0:42307:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644836992 00000100:00100000:2.0:1769754723.999885:0:42307:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644836992 00000100:00100000:2.0:1769754723.999957:0:42307:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 147 00000100:00100000:2.0:1769754723.999967:0:42307:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbb955bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:12a461d6-d4b3-45fe-9a31-84fa5ea63cfd+18:3320:x1855720644836992:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769754724.000057:0:42307:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbb955bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:12a461d6-d4b3-45fe-9a31-84fa5ea63cfd+18:3320:x1855720644836992:12345-0@lo:400:kworker.0 Request processed in 91us (7759us total) trans 0 rc 0/0 00000100:00100000:2.0:1769754724.000064:0:42307:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 147 00000100:00100000:3.0:1769754724.000083:0:3320:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb98566d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:12a461d6-d4b3-45fe-9a31-84fa5ea63cfd:3320:1855720644836992:0@lo:400:kworker.0 00010000:00010000:3.0:1769754724.052467:0:42301:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1769754724.117110:0:42347:0:(llog.c:797:llog_process_thread()) index: 8, lh_last_idx: 8 synced_idx: 8 lgh_last_idx: 8 00000040:00080000:0.0:1769754724.117119:0:42347:0:(llog_cat.c:828:llog_cat_process_common()) processing log [0x1:0x32:0x0] at index 8 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1769754724.117176:0:42347:0:(llog.c:854:llog_process_thread()) stop processing plain [0x1:0x32:0x0] index 5 count 5 00000040:00080000:0.0:1769754724.117185:0:42347:0:(llog.c:854:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 9 count 2 00000100:00100000:0.0:1769754724.117337:0:42347:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cbb8704a80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:42347:1855720644837760:192.168.202.6@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1769754724.117447:0:42347:0:(client.c:2690:ptlrpc_set_wait()) set ffff90cbb9621b40 going to sleep for 11 seconds 00000100:00100000:1.0:1769754724.130806:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:3.0:1769754724.130897:0:42315:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624209152 00000100:00100000:3.0:1769754724.130907:0:42315:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624209152 00000100:00080000:1.0:1769754724.130960:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1769754724.130986:0:42315:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 182 00000100:00100000:3.0:1769754724.130998:0:42315:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbaf97bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+13:49739:x1855720624209152:12345-192.168.202.6@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1769754724.131028:0:42315:0:(mdt_handler.c:7932:mdt_fid2path()) path get [0x200003ab1:0x14:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1769754724.131132:0:42315:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbaf97bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+13:49739:x1855720624209152:12345-192.168.202.6@tcp:53:lhsmtool_posix.0 Request processed in 137us (336us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754724.131140:0:42315:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 182 00000100:00100000:0.0:1769754724.131349:0:42347:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cbb8704a80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:42347:1855720644837760:192.168.202.6@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1769754724.145463:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:2.0:1769754724.146413:0:42314:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624209280 00000100:00100000:2.0:1769754724.146427:0:42314:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624209280 00000100:00100000:2.0:1769754724.146530:0:42314:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 183 00000100:00100000:2.0:1769754724.146542:0:42314:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbaf97b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+12:49739:x1855720624209280:12345-192.168.202.6@tcp:34:lhsmtool_posix.0 00010000:00010000:2.0:1769754724.146634:0:42314:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df244d lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.146653:0:42314:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df244d lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.146667:0:42314:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df244d lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.146693:0:42314:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100800/0x11e65116e9df2454 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x9753f962 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.146701:0:42314:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100800/0x11e65116e9df2454 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x9753f962 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.146728:0:42314:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100800/0x11e65116e9df2454 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x9753f962 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.146759:0:42314:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100600/0x11e65116e9df245b lrc: 3/1,0 mode: --/PR res: [0x200003ab1:0x14:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.146768:0:42314:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100600/0x11e65116e9df245b lrc: 3/1,0 mode: PR/PR res: [0x200003ab1:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.146775:0:42314:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100600/0x11e65116e9df245b lrc: 3/1,0 mode: PR/PR res: [0x200003ab1:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1769754724.146841:0:42314:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200003ab1:0x14:0x0]: returning size 11 00000004:00200000:2.0:1769754724.146847:0:42314:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1769754724.146862:0:42314:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100600/0x11e65116e9df245b lrc: 3/1,0 mode: PR/PR res: [0x200003ab1:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.146874:0:42314:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df244d lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.146881:0:42314:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df244d lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.146891:0:42314:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df244d lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.146896:0:42314:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff90cb92101400 already unused, calling callback (ffffffffc2101550) 00010000:00010000:2.0:1769754724.146900:0:42314:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df244d lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.146907:0:42314:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df244d lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.146925:0:42314:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff90cb87176c00) 00010000:00010000:2.0:1769754724.146927:0:42314:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769754724.146933:0:42314:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df244d lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.146939:0:42314:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df244d lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.146954:0:42314:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100800/0x11e65116e9df2454 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x9753f962 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.146959:0:42314:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100800/0x11e65116e9df2454 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x9753f962 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.146963:0:42314:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100800/0x11e65116e9df2454 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x9753f962 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.146967:0:42314:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff90cb92100800 already unused, calling callback (ffffffffc2101550) 00010000:00010000:2.0:1769754724.146970:0:42314:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100800/0x11e65116e9df2454 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x9753f962 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.146975:0:42314:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100800/0x11e65116e9df2454 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x9753f962 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.146981:0:42314:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x9753f962 (ffff90cb83cda800) 00010000:00010000:2.0:1769754724.146982:0:42314:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769754724.146985:0:42314:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100800/0x11e65116e9df2454 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x9753f962 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.147002:0:42314:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100800/0x11e65116e9df2454 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x9753f962 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.147027:0:42314:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100600/0x11e65116e9df245b lrc: 3/1,0 mode: PR/PR res: [0x200003ab1:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.147033:0:42314:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100600/0x11e65116e9df245b lrc: 2/0,0 mode: PR/PR res: [0x200003ab1:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.147038:0:42314:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100600/0x11e65116e9df245b lrc: 3/0,0 mode: PR/PR res: [0x200003ab1:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.147042:0:42314:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff90cb92100600 already unused, calling callback (ffffffffc2101550) 00010000:00010000:2.0:1769754724.147045:0:42314:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100600/0x11e65116e9df245b lrc: 3/0,0 mode: PR/PR res: [0x200003ab1:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.147050:0:42314:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100600/0x11e65116e9df245b lrc: 4/0,0 mode: PR/PR res: [0x200003ab1:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.147058:0:42314:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x14:0x0].0x0 (ffff90cb87176500) 00010000:00010000:2.0:1769754724.147060:0:42314:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769754724.147063:0:42314:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100600/0x11e65116e9df245b lrc: 2/0,0 mode: --/PR res: [0x200003ab1:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.147068:0:42314:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100600/0x11e65116e9df245b lrc: 0/0,0 mode: --/PR res: [0x200003ab1:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1769754724.147157:0:42314:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbaf97b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+12:49739:x1855720624209280:12345-192.168.202.6@tcp:34:lhsmtool_posix.0 Request processed in 617us (1704us total) trans 0 rc 0/0 00000100:00100000:2.0:1769754724.147164:0:42314:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 183 00000100:00100000:1.0:1769754724.154334:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:3.0:1769754724.154414:0:42315:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624209536 00000100:00100000:3.0:1769754724.154426:0:42315:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624209536 00000100:00100000:3.0:1769754724.154486:0:42315:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 184 00000100:00100000:3.0:1769754724.154541:0:42315:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbaf97a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+12:49739:x1855720624209536:12345-192.168.202.6@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1769754724.154561:0:42315:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769754724.154584:0:42315:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36800/0x11e65116e9df2462 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x2b94eb038208fd24 expref: -99 pid: 42315 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.155136:0:42315:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36c00/0x11e65116e9df2469 lrc: 3/1,0 mode: --/PR res: [0x200003ab1:0x14:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.155156:0:42315:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36c00/0x11e65116e9df2469 lrc: 3/1,0 mode: PR/PR res: [0x200003ab1:0x14:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.155168:0:42315:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36c00/0x11e65116e9df2469 lrc: 3/1,0 mode: PR/PR res: [0x200003ab1:0x14:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1769754724.155181:0:42315:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200003ab1:0x14:0x0]: returning size 11 00010000:00010000:3.0:1769754724.155324:0:42315:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36c00/0x11e65116e9df2469 lrc: 3/1,0 mode: PR/PR res: [0x200003ab1:0x14:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.155333:0:42315:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36c00/0x11e65116e9df2469 lrc: 2/0,0 mode: PR/PR res: [0x200003ab1:0x14:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.155340:0:42315:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36c00/0x11e65116e9df2469 lrc: 3/0,0 mode: PR/PR res: [0x200003ab1:0x14:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.155347:0:42315:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff90ca82c36c00 already unused, calling callback (ffffffffc2101550) 00010000:00010000:3.0:1769754724.155366:0:42315:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36c00/0x11e65116e9df2469 lrc: 3/0,0 mode: PR/PR res: [0x200003ab1:0x14:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.155376:0:42315:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36c00/0x11e65116e9df2469 lrc: 4/0,0 mode: PR/PR res: [0x200003ab1:0x14:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.155388:0:42315:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x14:0x0].0x0 (ffff90cb87176500) 00010000:00010000:3.0:1769754724.155392:0:42315:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1769754724.155397:0:42315:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36c00/0x11e65116e9df2469 lrc: 2/0,0 mode: --/PR res: [0x200003ab1:0x14:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.155403:0:42315:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36c00/0x11e65116e9df2469 lrc: 0/0,0 mode: --/PR res: [0x200003ab1:0x14:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.155453:0:42315:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36800/0x11e65116e9df2462 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd24 expref: 14 pid: 42315 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.155463:0:42315:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff90cb87176c00) 00010000:00010000:3.0:1769754724.155465:0:42315:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1769754724.155468:0:42315:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36800/0x11e65116e9df2462 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd24 expref: 14 pid: 42315 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.155477:0:42315:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff90ca82c36800, rc 0) 00000100:00100000:3.0:1769754724.155596:0:42315:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbaf97a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+13:49739:x1855720624209536:12345-192.168.202.6@tcp:101:lhsmtool_posix.0 Request processed in 1058us (1271us total) trans 34359738521 rc 301/301 00000100:00100000:3.0:1769754724.155605:0:42315:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 184 00000100:00100000:1.0:1769754724.167180:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:3.0:1769754724.167279:0:42315:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624209664 00000100:00100000:3.0:1769754724.167289:0:42315:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624209664 00000100:00100000:3.0:1769754724.167351:0:42315:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 185 00000100:00100000:3.0:1769754724.167361:0:42315:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbaf97aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+13:49739:x1855720624209664:12345-192.168.202.6@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1769754724.167411:0:42315:0:(mdt_handler.c:7932:mdt_fid2path()) path get [0x200003ab1:0x14:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1769754724.167883:0:42315:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbaf97aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+13:49739:x1855720624209664:12345-192.168.202.6@tcp:53:lhsmtool_posix.0 Request processed in 521us (711us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754724.167891:0:42315:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 185 00000100:00100000:0.0:1769754724.193695:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:3.0:1769754724.194303:0:42315:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624209920 00000100:00100000:3.0:1769754724.194315:0:42315:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624209920 00000100:00100000:3.0:1769754724.194426:0:42315:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 186 00000100:00100000:3.0:1769754724.194438:0:42315:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbb8704a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+13:49739:x1855720624209920:12345-192.168.202.6@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1769754724.194458:0:42315:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769754724.194462:0:42315:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1769754724.194488:0:42315:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81129e00/0x11e65116e9df239e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fcf3 expref: 13 pid: 42315 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.194538:0:42315:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff90cb83c8fd00) 00010000:00010000:3.0:1769754724.194548:0:42315:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1769754724.194563:0:42315:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36800/0x11e65116e9df2477 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x2b94eb038208fd2b expref: -99 pid: 42315 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1769754724.195142:0:42315:0:(osp_object.c:1657:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x106:0x0], index 1: 0 00010000:00010000:3.0:1769754724.195344:0:42315:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36c00/0x11e65116e9df247e lrc: 3/1,0 mode: --/PR res: [0x200003ab1:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.195359:0:42315:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36c00/0x11e65116e9df247e lrc: 3/1,0 mode: PR/PR res: [0x200003ab1:0x16:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.195369:0:42315:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36c00/0x11e65116e9df247e lrc: 3/1,0 mode: PR/PR res: [0x200003ab1:0x16:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.195446:0:42315:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36800/0x11e65116e9df2477 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd2b expref: 15 pid: 42315 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.195466:0:42315:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36c00/0x11e65116e9df247e lrc: 3/0,0 mode: PR/PR res: [0x200003ab1:0x16:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd2b expref: 14 pid: 42315 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.195474:0:42315:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x16:0x0].0x0 (ffff90cb87176100) 00010000:00010000:3.0:1769754724.195477:0:42315:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1769754724.195480:0:42315:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff90ca82c36c00, rc 0) 00000100:00100000:3.0:1769754724.195608:0:42315:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbb8704a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+14:49739:x1855720624209920:12345-192.168.202.6@tcp:101:lhsmtool_posix.0 Request processed in 1172us (1937us total) trans 34359738522 rc 0/0 00000100:00100000:3.0:1769754724.195616:0:42315:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 186 00000100:00100000:1.0:1769754724.203209:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:3.0:1769754724.203328:0:42315:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624210048 00000100:00100000:3.0:1769754724.203338:0:42315:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624210048 00000100:00100000:3.0:1769754724.203542:0:42315:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 187 00000100:00100000:3.0:1769754724.203553:0:42315:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbaf979c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+14:49739:x1855720624210048:12345-192.168.202.6@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1769754724.203578:0:42315:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769754724.203602:0:42315:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36800/0x11e65116e9df248c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x2b94eb038208fd32 expref: -99 pid: 42315 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.203688:0:42315:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c37e00/0x11e65116e9df2493 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.203703:0:42315:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c37e00/0x11e65116e9df2493 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.203715:0:42315:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c37e00/0x11e65116e9df2493 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1769754724.203804:0:42315:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 13824 00000004:00200000:3.0:1769754724.203808:0:42315:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 30 stripe count 1 MDT count 1 00000004:00010000:3.0:1769754724.203834:0:42315:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c37e00/0x11e65116e9df2493 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.203859:0:42315:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36800/0x11e65116e9df248c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd32 expref: 16 pid: 42315 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.203878:0:42315:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c37e00/0x11e65116e9df2493 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd32 expref: 15 pid: 42315 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.203886:0:42315:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff90cb87176e00) 00010000:00010000:3.0:1769754724.203889:0:42315:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1769754724.203892:0:42315:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff90ca82c37e00, rc 0) 00000100:00100000:3.0:1769754724.203957:0:42315:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbaf979c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+15:49739:x1855720624210048:12345-192.168.202.6@tcp:101:lhsmtool_posix.0 Request processed in 406us (759us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754724.203965:0:42315:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 187 00000100:00100000:0.0:1769754724.210459:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:3.0:1769754724.211415:0:42315:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624210304 00000100:00100000:3.0:1769754724.211425:0:42315:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624210304 00000100:00100000:2.0:1769754724.212468:0:42315:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 188 00000100:00100000:2.0:1769754724.212483:0:42315:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbb8706300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+15:49739:x1855720624210304:12345-192.168.202.6@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1769754724.212580:0:42315:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1769754724.212608:0:42315:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36c00/0x11e65116e9df247e lrc: 0/0,0 mode: --/PR res: [0x200003ab1:0x16:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd2b expref: 15 pid: 42315 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.212790:0:42315:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x16:0x0].0x0 (ffff90cb87176100) 00010000:00010000:2.0:1769754724.212888:0:42315:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c37e00/0x11e65116e9df2493 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd32 expref: 14 pid: 42315 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.212899:0:42315:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff90cb87176e00) 00010000:00010000:2.0:1769754724.212904:0:42315:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1769754724.212929:0:42315:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100600/0x11e65116e9df249a lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.212943:0:42315:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100600/0x11e65116e9df249a lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.212953:0:42315:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100600/0x11e65116e9df249a lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.212967:0:42315:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100800/0x11e65116e9df24a1 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xd8fed4fc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.212975:0:42315:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100800/0x11e65116e9df24a1 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xd8fed4fc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.212982:0:42315:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100800/0x11e65116e9df24a1 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xd8fed4fc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.213178:0:42315:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100600/0x11e65116e9df249a lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.213200:0:42315:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100600/0x11e65116e9df249a lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.213207:0:42315:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100600/0x11e65116e9df249a lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.213214:0:42315:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff90cb92100600 already unused, calling callback (ffffffffc2101550) 00010000:00010000:2.0:1769754724.213219:0:42315:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100600/0x11e65116e9df249a lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.213226:0:42315:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100600/0x11e65116e9df249a lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.213235:0:42315:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff90cb83cda800) 00010000:00010000:2.0:1769754724.213238:0:42315:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769754724.213242:0:42315:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100600/0x11e65116e9df249a lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.213248:0:42315:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100600/0x11e65116e9df249a lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.213263:0:42315:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100800/0x11e65116e9df24a1 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xd8fed4fc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.213269:0:42315:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100800/0x11e65116e9df24a1 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xd8fed4fc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.213276:0:42315:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100800/0x11e65116e9df24a1 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xd8fed4fc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.213281:0:42315:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff90cb92100800 already unused, calling callback (ffffffffc2101550) 00010000:00010000:2.0:1769754724.213290:0:42315:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100800/0x11e65116e9df24a1 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xd8fed4fc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.213297:0:42315:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100800/0x11e65116e9df24a1 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xd8fed4fc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.213305:0:42315:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xd8fed4fc (ffff90cb83cdac00) 00010000:00010000:2.0:1769754724.213308:0:42315:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769754724.213311:0:42315:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100800/0x11e65116e9df24a1 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xd8fed4fc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.213317:0:42315:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100800/0x11e65116e9df24a1 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xd8fed4fc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1769754724.213462:0:42315:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbb8706300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+13:49739:x1855720624210304:12345-192.168.202.6@tcp:36:lhsmtool_posix.0 Request processed in 983us (3010us total) trans 0 rc -2/-2 00000100:00100000:2.0:1769754724.213470:0:42315:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 188 00000100:00100000:0.0:1769754724.217027:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:2.0:1769754724.217117:0:42315:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624210432 00000100:00100000:2.0:1769754724.217127:0:42315:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624210432 00000100:00100000:2.0:1769754724.217188:0:42315:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 189 00000100:00100000:2.0:1769754724.217199:0:42315:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbb8707800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+13:49739:x1855720624210432:12345-192.168.202.6@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1769754724.217219:0:42315:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769754724.217243:0:42315:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df24a8 lrc: 2/0,0 mode: --/PR res: [0x200003ab1:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x2b94eb038208fd39 expref: -99 pid: 42315 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.217277:0:42315:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101600/0x11e65116e9df24af lrc: 3/1,0 mode: --/PR res: [0x200003ab1:0x16:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.217290:0:42315:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101600/0x11e65116e9df24af lrc: 3/1,0 mode: PR/PR res: [0x200003ab1:0x16:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.217320:0:42315:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101600/0x11e65116e9df24af lrc: 3/1,0 mode: PR/PR res: [0x200003ab1:0x16:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.217407:0:42315:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df24a8 lrc: 0/0,0 mode: --/PR res: [0x200003ab1:0x16:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd39 expref: 15 pid: 42315 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.217424:0:42315:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101600/0x11e65116e9df24af lrc: 3/0,0 mode: PR/PR res: [0x200003ab1:0x16:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd39 expref: 14 pid: 42315 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.217433:0:42315:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x16:0x0].0x0 (ffff90cb83cdae00) 00010000:00010000:2.0:1769754724.217436:0:42315:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1769754724.217439:0:42315:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff90cb92101600, rc 0) 00000100:00100000:2.0:1769754724.217544:0:42315:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbb8707800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+14:49739:x1855720624210432:12345-192.168.202.6@tcp:101:lhsmtool_posix.0 Request processed in 346us (526us total) trans 0 rc 0/0 00000100:00100000:2.0:1769754724.217551:0:42315:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 189 00000100:00100000:1.0:1769754724.234629:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:3.0:1769754724.243344:0:42314:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624210688 00000100:00100000:3.0:1769754724.243360:0:42314:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624210688 00000100:00100000:3.0:1769754724.243435:0:42314:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 190 00000100:00100000:3.0:1769754724.243446:0:42314:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbaf97a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+14:49739:x1855720624210688:12345-192.168.202.6@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1769754724.243558:0:42314:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36800/0x11e65116e9df24b6 lrc: 3/0,1 mode: --/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.243576:0:42314:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36800/0x11e65116e9df24b6 lrc: 3/0,1 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.243603:0:42314:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36800/0x11e65116e9df24b6 lrc: 3/0,1 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1769754724.244661:0:42314:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x30:0x0].5, 88 off8560 00000004:00080000:3.0:1769754724.244676:0:42314:0:(osp_sync.c:1841:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 1707232182030ns, next at 1708232178582ns, rc = 0 00000004:00010000:3.0:1769754724.245130:0:42314:0:(mdt_handler.c:4458:mdt_save_lock()) ### save lock request 0000000030f053aa reply state 00000000286d7f6f transno 34359738523 ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36800/0x11e65116e9df24b6 lrc: 3/0,1 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1769754724.245143:0:42314:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1769754724.245266:0:42314:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbaf97a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+16:49739:x1855720624210688:12345-192.168.202.6@tcp:36:lhsmtool_posix.0 Request processed in 1819us (10642us total) trans 34359738523 rc 0/0 00000100:00100000:3.0:1769754724.245275:0:42314:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 190 00000100:00100000:0.0:1769754724.256658:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00010000:00010000:1.0:1769754724.256791:0:3305:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36800/0x11e65116e9df24b6 lrc: 3/0,1 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754724.256811:0:3305:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36800/0x11e65116e9df24b6 lrc: 2/0,0 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754724.256818:0:3305:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36800/0x11e65116e9df24b6 lrc: 3/0,0 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754724.256825:0:3305:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff90ca82c36800 already unused, calling callback (ffffffffc2101550) 00010000:00010000:1.0:1769754724.256829:0:3305:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36800/0x11e65116e9df24b6 lrc: 3/0,0 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754724.256838:0:3305:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36800/0x11e65116e9df24b6 lrc: 4/0,0 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754724.256854:0:3305:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x16:0x0].0x0 (ffff90cb83cdae00) 00010000:00010000:1.0:1769754724.256857:0:3305:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1769754724.256862:0:3305:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36800/0x11e65116e9df24b6 lrc: 2/0,0 mode: --/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754724.256868:0:3305:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36800/0x11e65116e9df24b6 lrc: 0/0,0 mode: --/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1769754724.259232:0:42315:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624210816 00000100:00100000:2.0:1769754724.259250:0:42315:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624210816 00000100:00100000:2.0:1769754724.259330:0:42315:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 191 00000100:00100000:2.0:1769754724.259341:0:42315:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbb8706680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+15:49739:x1855720624210816:12345-192.168.202.6@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1769754724.259368:0:42315:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769754724.259661:0:42315:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df24bd lrc: 2/0,0 mode: --/CR res: [0x200003ab1:0x16:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x2b94eb038208fd40 expref: -99 pid: 42315 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.259720:0:42315:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df24bd lrc: 3/0,0 mode: CR/CR res: [0x200003ab1:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd40 expref: 16 pid: 42315 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.260044:0:42315:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df24bd lrc: 3/0,0 mode: CR/CR res: [0x200003ab1:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd40 expref: 16 pid: 42315 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.260087:0:42315:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x16:0x0].0x0 (ffff90cb83cdae00) 00010000:00010000:2.0:1769754724.260090:0:42315:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1769754724.260094:0:42315:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff90cb92101400, rc 0) 00000100:00100000:2.0:1769754724.260210:0:42315:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbb8706680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+16:49739:x1855720624210816:12345-192.168.202.6@tcp:101:lhsmtool_posix.0 Request processed in 869us (3560us total) trans 0 rc 0/0 00000100:00100000:2.0:1769754724.260218:0:42315:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 191 00000100:00100000:0.0:1769754724.272154:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:3.0:1769754724.272281:0:42938:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624210944 00000100:00100000:3.0:1769754724.272292:0:42938:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624210944 00000100:00080000:1.0:1769754724.272427:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1769754724.272654:0:42938:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 49 00000100:00100000:3.0:1769754724.272667:0:42938:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbb8705c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+6:2371:x1855720624210944:12345-192.168.202.6@tcp:2:lhsmtool_posix.0 00000100:00100000:3.0:1769754724.273035:0:42938:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbb8705c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+7:2371:x1855720624210944:12345-192.168.202.6@tcp:2:lhsmtool_posix.0 Request processed in 370us (890us total) trans 34359738380 rc 0/0 00000100:00100000:3.0:1769754724.273044:0:42938:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 49 00000100:00100000:1.0:1769754724.279338:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:2.0:1769754724.279559:0:42316:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624211200 00000100:00100000:2.0:1769754724.279570:0:42316:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624211200 00000100:00100000:2.0:1769754724.279630:0:42316:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 64 00000100:00100000:2.0:1769754724.279641:0:42316:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbaf979f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+16:49739:x1855720624211200:12345-192.168.202.6@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1769754724.280332:0:42316:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbaf979f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+17:49739:x1855720624211200:12345-192.168.202.6@tcp:35:lhsmtool_posix.0 Request processed in 689us (1001us total) trans 34359738524 rc 0/0 00000100:00100000:0.0:1769754724.280341:0:42316:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 64 00000100:00100000:1.0:1769754724.289060:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:2.0:1769754724.289963:0:42315:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624211456 00000100:00100000:2.0:1769754724.289974:0:42315:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624211456 00000100:00100000:2.0:1769754724.290044:0:42315:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 192 00000100:00100000:2.0:1769754724.290055:0:42315:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbaf97b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+17:49739:x1855720624211456:12345-192.168.202.6@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1769754724.290175:0:42315:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbaf97b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+17:49739:x1855720624211456:12345-192.168.202.6@tcp:57:lhsmtool_posix.0 Request processed in 121us (1124us total) trans 0 rc 0/0 00000100:00100000:2.0:1769754724.290184:0:42315:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 192 00000100:00100000:2.0:1769754724.299614:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:3.0:1769754724.299868:0:42314:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624211584 00000100:00100000:3.0:1769754724.299887:0:42314:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624211584 00000100:00100000:3.0:1769754724.299944:0:42314:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 193 00000100:00100000:3.0:1769754724.299955:0:42314:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90ca86de3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+17:49739:x1855720624211584:12345-192.168.202.6@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1769754724.299976:0:42314:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769754724.300377:0:42314:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c37800/0x11e65116e9df24c4 lrc: 2/0,0 mode: --/CR res: [0x200003ab1:0x16:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x2b94eb038208fd47 expref: -99 pid: 42314 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754724.300564:0:42314:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81128e00/0x11e65116e9df24cb lrc: 3/1,0 mode: --/PR res: [0x200003ab1:0x16:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754724.300579:0:42314:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81128e00/0x11e65116e9df24cb lrc: 3/1,0 mode: PR/PR res: [0x200003ab1:0x16:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754724.300591:0:42314:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81128e00/0x11e65116e9df24cb lrc: 3/1,0 mode: PR/PR res: [0x200003ab1:0x16:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1769754724.300629:0:42314:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1769754724.300646:0:42314:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81128e00/0x11e65116e9df24cb lrc: 3/1,0 mode: PR/PR res: [0x200003ab1:0x16:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754724.300671:0:42314:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c37800/0x11e65116e9df24c4 lrc: 0/0,0 mode: --/CR res: [0x200003ab1:0x16:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd47 expref: 19 pid: 42314 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754724.300700:0:42314:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81128e00/0x11e65116e9df24cb lrc: 3/0,0 mode: PR/PR res: [0x200003ab1:0x16:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd47 expref: 18 pid: 42314 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754724.300707:0:42314:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x16:0x0].0x0 (ffff90cb83cdae00) 00010000:00010000:0.0:1769754724.300710:0:42314:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1769754724.300714:0:42314:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff90ca81128e00, rc 0) 00000100:00100000:0.0:1769754724.300810:0:42314:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90ca86de3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+18:49739:x1855720624211584:12345-192.168.202.6@tcp:101:lhsmtool_posix.0 Request processed in 858us (1206us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754724.300818:0:42314:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 193 00000100:00100000:0.0:1769754724.312751:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:3.0:1769754724.313364:0:42938:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624211712 00000100:00100000:3.0:1769754724.313375:0:42938:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624211712 00000100:00100000:3.0:1769754724.313436:0:42938:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 50 00000100:00100000:3.0:1769754724.313446:0:42938:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbb8704380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+7:2371:x1855720624211712:12345-192.168.202.6@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1769754724.313465:0:42938:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769754724.313488:0:42938:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f489c14b/0x11e65116e9df24d2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x106:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x2b94eb038208fd4e expref: -99 pid: 42938 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1769754724.313566:0:42938:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x280000400:0x106:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x697c5065, blocks: 0x1 00010000:00010000:3.0:1769754724.313585:0:42938:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769754724.313588:0:42938:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769754724.313596:0:42938:0:(ldlm_resource.c:1714:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f489c14b/0x11e65116e9df24d2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x106:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd4e expref: 8 pid: 42938 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.313614:0:42938:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f489c14b/0x11e65116e9df24d2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x106:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd4e expref: 8 pid: 42938 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.313627:0:42938:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff90ca82c37000, rc 0) 00000100:00100000:3.0:1769754724.313690:0:42938:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbb8704380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+8:2371:x1855720624211712:12345-192.168.202.6@tcp:101:lhsmtool_posix.0 Request processed in 245us (956us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754724.313697:0:42938:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 50 00000100:00100000:1.0:1769754724.319757:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:2.0:1769754724.319888:0:42315:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624211968 00000100:00100000:2.0:1769754724.319901:0:42315:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624211968 00000100:00100000:2.0:1769754724.319961:0:42315:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 194 00000100:00100000:2.0:1769754724.319971:0:42315:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbaf979180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+18:49739:x1855720624211968:12345-192.168.202.6@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1769754724.320082:0:42315:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbaf979180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+18:49739:x1855720624211968:12345-192.168.202.6@tcp:57:lhsmtool_posix.0 Request processed in 112us (335us total) trans 0 rc 0/0 00000100:00100000:2.0:1769754724.320107:0:42315:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 194 00000100:00100000:2.0:1769754724.325432:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:0.0:1769754724.326277:0:43451:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624212096 00000100:00100000:0.0:1769754724.326288:0:43451:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624212096 00000100:00100000:0.0:1769754724.326402:0:43451:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 51 00000100:00100000:0.0:1769754724.326414:0:43451:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90ca86de2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+8:49739:x1855720624212096:12345-192.168.202.6@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1769754724.326433:0:43451:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769754724.326452:0:43451:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e7c3556f/0x11e65116e9df24d9 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x2b94eb038208fd55 expref: -99 pid: 43451 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754724.326485:0:43451:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f489c14b/0x11e65116e9df24d2 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd4e expref: 9 pid: 42938 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754724.326528:0:43451:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769754724.326530:0:43451:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1769754724.326535:0:43451:0:(ldlm_resource.c:1714:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e7c3556f/0x11e65116e9df24d9 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd55 expref: 9 pid: 43451 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754724.326607:0:43451:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f489c14b/0x11e65116e9df24d2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd4e expref: 9 pid: 42938 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754724.326622:0:43451:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f489c14b/0x11e65116e9df24d2 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd4e expref: 9 pid: 42938 timeout: 1807 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1769754724.326637:0:43451:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb8718b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:43451:1855720644837888:192.168.202.6@tcp:104: 00000100:00100000:0.0:1769754724.326750:0:43451:0:(client.c:2690:ptlrpc_set_wait()) set ffff90cbb9621b40 going to sleep for 11 seconds 00000100:00100000:1.0:1769754724.333437:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:0.0:1769754724.333703:0:43451:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb8718b480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:43451:1855720644837888:192.168.202.6@tcp:104: 00010000:00010000:0.0:1769754724.333753:0:43451:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e7c3556f/0x11e65116e9df24d9 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd55 expref: 9 pid: 43451 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754724.333789:0:43451:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff90ca81129e00, rc 0) 00000100:00100000:0.0:1769754724.333869:0:43451:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90ca86de2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+9:49739:x1855720624212096:12345-192.168.202.6@tcp:101:lhsmtool_posix.0 Request processed in 7457us (8444us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754724.334638:0:43451:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 51 00000100:00100000:3.0:1769754724.335657:0:42299:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624212224 00000100:00100000:3.0:1769754724.335670:0:42299:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624212224 00010000:00010000:3.0:1769754724.335711:0:42299:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f489c14b/0x11e65116e9df24d2 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd4e expref: 9 pid: 42938 timeout: 1807 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1769754724.336166:0:42299:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 33 00000100:00100000:3.0:1769754724.336176:0:42299:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90ca88cc4050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+9:2371:x1855720624212224:12345-192.168.202.6@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769754724.336192:0:42299:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769754724.336215:0:42299:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f489c14b/0x11e65116e9df24d2 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x106:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd4e expref: 9 pid: 42938 timeout: 1807 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.336229:0:42299:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f489c14b/0x11e65116e9df24d2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x106:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd4e expref: 9 pid: 42938 timeout: 1807 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.336251:0:42299:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f489c14b/0x11e65116e9df24d2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x106:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd4e expref: 9 pid: 42938 timeout: 1807 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.336266:0:42299:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769754724.336269:0:42299:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769754724.336276:0:42299:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e7c3556f/0x11e65116e9df24d9 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd55 expref: 8 pid: 43451 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.336284:0:42299:0:(ldlm_resource.c:1714:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e7c3556f/0x11e65116e9df24d9 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd55 expref: 8 pid: 43451 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.336317:0:42299:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000e7c3556f/0x11e65116e9df24d9 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd55 expref: 8 pid: 43451 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1769754724.336330:0:42299:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb87861880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:42299:1855720644838016:192.168.202.6@tcp:105: 00000100:00100000:3.0:1769754724.336409:0:42299:0:(client.c:2690:ptlrpc_set_wait()) set ffff90cbb4851780 going to sleep for 11 seconds 00000100:00100000:3.0:1769754724.350452:0:42299:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb87861880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:42299:1855720644838016:192.168.202.6@tcp:105: 00010000:00010000:3.0:1769754724.350479:0:42299:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769754724.350584:0:42299:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90ca88cc4050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+8:2371:x1855720624212224:12345-192.168.202.6@tcp:103:ldlm_bl.0 Request processed in 14409us (17157us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754724.350603:0:42299:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 33 00000100:00100000:2.0:1769754724.351230:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:3.0:1769754724.351720:0:42942:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624212352 00000100:00100000:3.0:1769754724.351729:0:42942:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624212352 00002000:00100000:3.0:1769754724.351789:0:42942:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x106:0x0] (0->10) req@ffff90ca86de1c00 x1855720624212352/t0(0) o4->44b15f98-ec2b-44ce-9a84-650ae6cf82b2@192.168.202.6@tcp:5/0 lens 504/0 e 0 to 0 dl 1769754735 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769754724.351799:0:42942:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff90ca86de1c00 with x1855720624212352 ext(0->10) 00002000:00010000:3.0:1769754724.351805:0:42942:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff90ca86de1c00 00000100:00100000:3.0:1769754724.351869:0:42942:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 5 00000100:00100000:3.0:1769754724.351889:0:42942:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90ca86de1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+8:2372:x1855720624212352:12345-192.168.202.6@tcp:4:lhsmtool_posix.0 00000100:00100000:3.0:1769754724.352693:0:42942:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90ca86de1c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+10:2372:x1855720624212352:12345-192.168.202.6@tcp:4:lhsmtool_posix.0 Request processed in 810us (1459us total) trans 34359738381 rc 0/0 00000100:00100000:3.0:1769754724.352703:0:42942:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 5 00002000:00100000:3.0:1769754724.352712:0:42942:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x106:0x0] (0->10) req@ffff90ca86de1c00 x1855720624212352/t34359738381(0) o4->44b15f98-ec2b-44ce-9a84-650ae6cf82b2@192.168.202.6@tcp:5/0 lens 504/448 e 0 to 0 dl 1769754735 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769754724.352721:0:42942:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff90ca86de1c00 with x1855720624212352 ext(0->10) 00002000:00010000:3.0:1769754724.352736:0:42942:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff90ca86de1c00 00000100:00100000:2.0:1769754724.357833:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:0.0:1769754724.358019:0:42314:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624212480 00000100:00100000:0.0:1769754724.358030:0:42314:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624212480 00000100:00100000:0.0:1769754724.358249:0:42314:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 195 00000100:00100000:0.0:1769754724.358262:0:42314:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90ca86de2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+18:49739:x1855720624212480:12345-192.168.202.6@tcp:44:lhsmtool_posix.0 00000001:00080000:3.0:1769754724.451195:0:44975:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 34359738524 is committed 00000004:00080000:3.0:1769754724.451215:0:44975:0:(osp_sync.c:1806:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 5 committed 00000100:00100000:3.0:1769754724.451399:0:3318:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb87860000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3318:1855720644838144:0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1769754724.451465:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769754724.451563:0:43451:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644838144 00000100:00100000:3.0:1769754724.451571:0:43451:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644838144 00000100:00100000:3.0:1769754724.451815:0:43451:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 52 00000100:00100000:3.0:1769754724.451826:0:43451:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb87861880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3318:x1855720644838144:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:3.0:1769754724.451977:0:43451:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000400:0x106:0x0] updating lvb size from disk: 0 -> 11 00002000:00010000:3.0:1769754724.451980:0:43451:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000400:0x106:0x0] updating lvb mtime from disk: 0 -> 1769754725 00002000:00010000:3.0:1769754724.451982:0:43451:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000400:0x106:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:3.0:1769754724.451984:0:43451:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000400:0x106:0x0] updating lvb ctime from disk: 1769754725 -> 1769754725 00002000:00010000:3.0:1769754724.451985:0:43451:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000400:0x106:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:3.0:1769754724.452039:0:43451:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb87861880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3318:x1855720644838144:12345-0@lo:2:osp-syn-1-0.0 Request processed in 215us (572us total) trans 34359738382 rc 0/0 00000100:00100000:3.0:1769754724.452047:0:43451:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 52 00000100:00100000:1.0:1769754724.452180:0:42314:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90ca86de2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+14:49739:x1855720624212480:12345-192.168.202.6@tcp:44:lhsmtool_posix.0 Request processed in 93918us (94354us total) trans 0 rc 0/0 00000100:00100000:1.0:1769754724.452189:0:42314:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 195 00000004:00080000:2.0:1769754724.452337:0:3318:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff90cb87860000/2, rc 0, transno 34359738382 00000100:00100000:2.0:1769754724.452344:0:3318:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb87860000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3318:1855720644838144:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1769754724.455740:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:3.0:1769754724.456265:0:43451:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624212608 00000100:00100000:3.0:1769754724.456277:0:43451:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624212608 00000100:00100000:3.0:1769754724.456339:0:43451:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 53 00000100:00100000:3.0:1769754724.456349:0:43451:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbaf978380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+10:2371:x1855720624212608:12345-192.168.202.6@tcp:16:lhsmtool_posix.0 00000001:00080000:0.0:1769754724.496721:0:7447:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 34359738382 is committed 00000001:00080000:0.0:1769754724.496740:0:7447:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 34359738381 is committed 00000100:00100000:1.0:1769754724.497174:0:43451:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbaf978380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+7:2371:x1855720624212608:12345-192.168.202.6@tcp:16:lhsmtool_posix.0 Request processed in 40819us (41438us total) trans 0 rc 0/0 00000100:00100000:1.0:1769754724.497186:0:43451:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 53 00000100:00100000:0.0:1769754724.500261:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:1.0:1769754724.500480:0:42314:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624212864 00000100:00100000:1.0:1769754724.500520:0:42314:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624212864 00000100:00100000:1.0:1769754724.500582:0:42314:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 196 00000100:00100000:1.0:1769754724.500593:0:42314:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb8718b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+14:49739:x1855720624212864:12345-192.168.202.6@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1769754724.500647:0:42314:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1769754724.500675:0:42314:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81128e00/0x11e65116e9df24cb lrc: 0/0,0 mode: --/PR res: [0x200003ab1:0x16:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd47 expref: 14 pid: 42314 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754724.500694:0:42314:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x16:0x0].0x0 (ffff90cb83cdae00) 00010000:00010000:1.0:1769754724.500699:0:42314:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1769754724.500742:0:42314:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d6e00/0x11e65116e9df24e0 lrc: 3/0,1 mode: --/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754724.500756:0:42314:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d6e00/0x11e65116e9df24e0 lrc: 3/0,1 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754724.500765:0:42314:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d6e00/0x11e65116e9df24e0 lrc: 3/0,1 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754724.500818:0:42314:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d6e00/0x11e65116e9df24e0 lrc: 3/0,1 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754724.500826:0:42314:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d6e00/0x11e65116e9df24e0 lrc: 2/0,0 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754724.500834:0:42314:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d6e00/0x11e65116e9df24e0 lrc: 3/0,0 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754724.500840:0:42314:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff90ca887d6e00 already unused, calling callback (ffffffffc2101550) 00010000:00010000:1.0:1769754724.500845:0:42314:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d6e00/0x11e65116e9df24e0 lrc: 3/0,0 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754724.500852:0:42314:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d6e00/0x11e65116e9df24e0 lrc: 4/0,0 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754724.500861:0:42314:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x16:0x0].0x0 (ffff90cb83cdae00) 00010000:00010000:1.0:1769754724.500864:0:42314:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1769754724.500868:0:42314:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d6e00/0x11e65116e9df24e0 lrc: 2/0,0 mode: --/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754724.500873:0:42314:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d6e00/0x11e65116e9df24e0 lrc: 0/0,0 mode: --/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1769754724.500977:0:42314:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb8718b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+13:49739:x1855720624212864:12345-192.168.202.6@tcp:36:lhsmtool_posix.0 Request processed in 385us (727us total) trans 0 rc 0/0 00000100:00100000:1.0:1769754724.500985:0:42314:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 196 00000100:00100000:2.0:1769754724.506997:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:1.0:1769754724.507102:0:43451:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624212992 00000100:00100000:1.0:1769754724.507112:0:43451:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624212992 00000100:00100000:1.0:1769754724.507216:0:43451:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 54 00000100:00100000:1.0:1769754724.507242:0:43451:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90ca8cca7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+7:2372:x1855720624212992:12345-192.168.202.6@tcp:2:lhsmtool_posix.0 00002000:00010000:1.0:1769754724.507765:0:43451:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x280000400:0x106:0x0] updating lvb size from disk: 11 -> 11 00002000:00010000:1.0:1769754724.507770:0:43451:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x280000400:0x106:0x0] updating lvb mtime from disk: 1769754725 -> 1769754714 00002000:00010000:1.0:1769754724.507773:0:43451:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x280000400:0x106:0x0] updating lvb atime from disk: 0 -> 1769754715 00002000:00010000:1.0:1769754724.507788:0:43451:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x280000400:0x106:0x0] updating lvb ctime from disk: 1769754725 -> 1769754715 00002000:00010000:1.0:1769754724.507790:0:43451:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x280000400:0x106:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:1.0:1769754724.507917:0:43451:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90ca8cca7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+8:2372:x1855720624212992:12345-192.168.202.6@tcp:2:lhsmtool_posix.0 Request processed in 676us (929us total) trans 34359738383 rc 0/0 00000100:00100000:1.0:1769754724.507927:0:43451:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 54 00000100:00100000:2.0:1769754724.517261:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:1.0:1769754724.517413:0:42315:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624213248 00000100:00100000:1.0:1769754724.517425:0:42315:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624213248 00000100:00100000:1.0:1769754724.517485:0:42315:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 197 00000100:00100000:1.0:1769754724.517525:0:42315:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90ca8cca5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+13:49739:x1855720624213248:12345-192.168.202.6@tcp:55:lhsmtool_posix.0 00010000:00010000:1.0:1769754724.517597:0:42315:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d6e00/0x11e65116e9df24e7 lrc: 3/0,1 mode: --/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754724.517615:0:42315:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101600/0x11e65116e9df24af lrc: 2/0,0 mode: PR/PR res: [0x200003ab1:0x16:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd39 expref: 13 pid: 42315 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754724.517626:0:42315:0:(ldlm_resource.c:1714:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d6e00/0x11e65116e9df24e7 lrc: 4/0,1 mode: --/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754724.517664:0:42315:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101600/0x11e65116e9df24af lrc: 3/0,0 mode: PR/PR res: [0x200003ab1:0x16:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd39 expref: 13 pid: 42315 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754724.517679:0:42315:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101600/0x11e65116e9df24af lrc: 4/0,0 mode: PR/PR res: [0x200003ab1:0x16:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd39 expref: 13 pid: 42315 timeout: 1807 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1769754724.517718:0:42315:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cbaf978e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:42315:1855720644838272:192.168.202.6@tcp:104: 00000100:00100000:1.0:1769754724.517834:0:42315:0:(client.c:2690:ptlrpc_set_wait()) set ffff90cb915346c0 going to sleep for 11 seconds 00000100:00100000:1.0:1769754724.520727:0:42315:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cbaf978e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:42315:1855720644838272:192.168.202.6@tcp:104: 00010000:00010000:1.0:1769754724.520759:0:42315:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d6e00/0x11e65116e9df24e7 lrc: 3/0,1 mode: --/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1769754724.521690:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:3.0:1769754724.521920:0:42299:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624213376 00000100:00100000:3.0:1769754724.521930:0:42299:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624213376 00010000:00010000:3.0:1769754724.521970:0:42299:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101600/0x11e65116e9df24af lrc: 4/0,0 mode: PR/PR res: [0x200003ab1:0x16:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd39 expref: 14 pid: 42315 timeout: 1807 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1769754724.522021:0:42299:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 34 00000100:00100000:3.0:1769754724.522032:0:42299:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbbf87b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+14:2371:x1855720624213376:12345-192.168.202.6@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769754724.522052:0:42299:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769754724.522059:0:42299:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101600/0x11e65116e9df24af lrc: 4/0,0 mode: PR/PR res: [0x200003ab1:0x16:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd39 expref: 14 pid: 42315 timeout: 1807 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.522089:0:42299:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101600/0x11e65116e9df24af lrc: 3/0,0 mode: PR/PR res: [0x200003ab1:0x16:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd39 expref: 14 pid: 42315 timeout: 1807 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.522112:0:42299:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101600/0x11e65116e9df24af lrc: 0/0,0 mode: --/PR res: [0x200003ab1:0x16:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd39 expref: 14 pid: 42315 timeout: 1807 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.522130:0:42299:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x16:0x0].0x0 (ffff90cb83cdae00) 00010000:00010000:3.0:1769754724.522136:0:42299:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d6e00/0x11e65116e9df24e7 lrc: 3/0,1 mode: --/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.522147:0:42299:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d6e00/0x11e65116e9df24e7 lrc: 3/0,1 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.522156:0:42299:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d6e00/0x11e65116e9df24e7 lrc: 4/0,1 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.522191:0:42299:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769754724.522245:0:42299:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbbf87b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+13:2371:x1855720624213376:12345-192.168.202.6@tcp:103:ldlm_bl.0 Request processed in 213us (557us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754724.522254:0:42299:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 34 00010000:00010000:2.0:1769754724.522409:0:42315:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d6e00/0x11e65116e9df24e7 lrc: 3/0,1 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.522418:0:42315:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d6e00/0x11e65116e9df24e7 lrc: 3/0,1 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.522449:0:42315:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d6e00/0x11e65116e9df24e7 lrc: 3/0,1 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.522457:0:42315:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d6e00/0x11e65116e9df24e7 lrc: 2/0,0 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.522463:0:42315:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d6e00/0x11e65116e9df24e7 lrc: 3/0,0 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.522469:0:42315:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff90ca887d6e00 already unused, calling callback (ffffffffc2101550) 00010000:00010000:2.0:1769754724.522474:0:42315:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d6e00/0x11e65116e9df24e7 lrc: 3/0,0 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.522483:0:42315:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d6e00/0x11e65116e9df24e7 lrc: 4/0,0 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.522535:0:42315:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x16:0x0].0x0 (ffff90cb83cdae00) 00010000:00010000:2.0:1769754724.522538:0:42315:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1769754724.522542:0:42315:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d6e00/0x11e65116e9df24e7 lrc: 2/0,0 mode: --/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.522548:0:42315:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d6e00/0x11e65116e9df24e7 lrc: 0/0,0 mode: --/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1769754724.522598:0:42315:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90ca8cca5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+12:49739:x1855720624213248:12345-192.168.202.6@tcp:55:lhsmtool_posix.0 Request processed in 5103us (5348us total) trans 0 rc -22/-22 00000100:00100000:2.0:1769754724.522608:0:42315:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 197 00000100:00100000:2.0:1769754724.525592:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:1.0:1769754724.525788:0:43451:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624213504 00000100:00100000:1.0:1769754724.525799:0:43451:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624213504 00000100:00100000:1.0:1769754724.525903:0:43451:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 55 00000100:00100000:1.0:1769754724.525916:0:43451:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbbf86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+8:2372:x1855720624213504:12345-192.168.202.6@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1769754724.525966:0:43451:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000004ae526ef/0x11e65116e9df24ee lrc: 3/1,0 mode: --/PR res: [0x280000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 43451 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754724.525991:0:43451:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e7c3556f/0x11e65116e9df24d9 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd55 expref: 8 pid: 43451 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754724.526001:0:43451:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769754724.526003:0:43451:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:1.0:1769754724.526007:0:43451:0:(ldlm_resource.c:1714:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004ae526ef/0x11e65116e9df24ee lrc: 4/1,0 mode: --/PR res: [0x280000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 43451 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754724.526050:0:43451:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e7c3556f/0x11e65116e9df24d9 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd55 expref: 8 pid: 43451 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754724.526066:0:43451:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e7c3556f/0x11e65116e9df24d9 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd55 expref: 8 pid: 43451 timeout: 1807 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1769754724.526099:0:43451:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cbaf978e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:43451:1855720644838400:192.168.202.6@tcp:104: 00000100:00100000:1.0:1769754724.526189:0:43451:0:(client.c:2690:ptlrpc_set_wait()) set ffff90cb915346c0 going to sleep for 11 seconds 00000100:00100000:0.0:1769754724.533460:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:1.0:1769754724.533839:0:43451:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cbaf978e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:43451:1855720644838400:192.168.202.6@tcp:104: 00010000:00010000:1.0:1769754724.533871:0:43451:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000004ae526ef/0x11e65116e9df24ee lrc: 3/1,0 mode: --/PR res: [0x280000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 43451 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1769754724.534451:0:42300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624213632 00000100:00100000:2.0:1769754724.534460:0:42300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624213632 00010000:00010000:2.0:1769754724.534527:0:42300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e7c3556f/0x11e65116e9df24d9 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd55 expref: 9 pid: 43451 timeout: 1807 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1769754724.534571:0:42300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 35 00000100:00100000:2.0:1769754724.534581:0:42300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb87188e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+9:2371:x1855720624213632:12345-192.168.202.6@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769754724.534596:0:42300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769754724.534618:0:42300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e7c3556f/0x11e65116e9df24d9 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x106:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd55 expref: 9 pid: 43451 timeout: 1807 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.534631:0:42300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e7c3556f/0x11e65116e9df24d9 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x106:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd55 expref: 9 pid: 43451 timeout: 1807 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.534654:0:42300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e7c3556f/0x11e65116e9df24d9 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x106:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd55 expref: 9 pid: 43451 timeout: 1807 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.534682:0:42300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769754724.534684:0:42300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769754724.534690:0:42300:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000004ae526ef/0x11e65116e9df24ee lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 43451 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.534700:0:42300:0:(ldlm_resource.c:1714:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004ae526ef/0x11e65116e9df24ee lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 43451 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.534728:0:42300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769754724.535265:0:42300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb87188e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+8:2371:x1855720624213632:12345-192.168.202.6@tcp:103:ldlm_bl.0 Request processed in 682us (1811us total) trans 0 rc 0/0 00000100:00100000:2.0:1769754724.535275:0:42300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 35 00010000:00010000:0.0:1769754724.536719:0:43451:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000004ae526ef/0x11e65116e9df24ee lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x106:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 43451 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754724.536736:0:43451:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004ae526ef/0x11e65116e9df24ee lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x106:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 43451 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754724.536792:0:43451:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000004ae526ef/0x11e65116e9df24ee lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x106:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 43451 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754724.536801:0:43451:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000004ae526ef/0x11e65116e9df24ee lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x106:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 43451 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754724.536809:0:43451:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000004ae526ef/0x11e65116e9df24ee lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x106:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 43451 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754724.536816:0:43451:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff90ca887d7e00 already unused, calling callback (ffffffffc16a4360) 00010000:00010000:0.0:1769754724.536821:0:43451:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000004ae526ef/0x11e65116e9df24ee lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x106:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 43451 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754724.536829:0:43451:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000004ae526ef/0x11e65116e9df24ee lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x106:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 43451 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754724.536847:0:43451:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000004ae526ef/0x11e65116e9df24ee lrc: 2/0,0 mode: --/PR res: [0x280000400:0x106:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 43451 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754724.536853:0:43451:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004ae526ef/0x11e65116e9df24ee lrc: 0/0,0 mode: --/PR res: [0x280000400:0x106:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 43451 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1769754724.537275:0:43451:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbbf86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+7:2372:x1855720624213504:12345-192.168.202.6@tcp:1:lhsmtool_posix.0 Request processed in 11359us (11691us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754724.537282:0:43451:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 55 00000100:00100000:0.0:1769754724.541442:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:1.0:1769754724.542196:0:42315:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624213888 00000100:00100000:1.0:1769754724.542207:0:42315:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624213888 00000100:00100000:1.0:1769754724.542311:0:42315:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 198 00000100:00100000:1.0:1769754724.542322:0:42315:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb87189180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+12:49739:x1855720624213888:12345-192.168.202.6@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1769754724.542685:0:42315:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d7a00/0x11e65116e9df24f5 lrc: 3/0,1 mode: --/EX res: [0x200003ab1:0x16:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754724.542704:0:42315:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df24bd lrc: 2/0,0 mode: CR/CR res: [0x200003ab1:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd40 expref: 13 pid: 42315 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754724.542714:0:42315:0:(ldlm_resource.c:1714:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d7a00/0x11e65116e9df24f5 lrc: 4/0,1 mode: --/EX res: [0x200003ab1:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754724.542753:0:42315:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df24bd lrc: 3/0,0 mode: CR/CR res: [0x200003ab1:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd40 expref: 13 pid: 42315 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754724.543233:0:42315:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df24bd lrc: 4/0,0 mode: CR/CR res: [0x200003ab1:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd40 expref: 13 pid: 42315 timeout: 1807 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1769754724.543269:0:42315:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cbaf978e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:42315:1855720644838528:192.168.202.6@tcp:104: 00000100:00100000:1.0:1769754724.543371:0:42315:0:(client.c:2690:ptlrpc_set_wait()) set ffff90cb915346c0 going to sleep for 11 seconds 00000100:00100000:3.0:1769754724.547474:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:2.0:1769754724.547667:0:42300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624214016 00000100:00100000:2.0:1769754724.547678:0:42300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624214016 00010000:00010000:2.0:1769754724.547719:0:42300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df24bd lrc: 5/0,0 mode: CR/CR res: [0x200003ab1:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd40 expref: 14 pid: 42315 timeout: 1807 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1769754724.547790:0:42300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 36 00000100:00100000:2.0:1769754724.547801:0:42300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb87860700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+14:2372:x1855720624214016:12345-192.168.202.6@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769754724.547818:0:42300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769754724.547825:0:42300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df24bd lrc: 5/0,0 mode: CR/CR res: [0x200003ab1:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd40 expref: 14 pid: 42315 timeout: 1807 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.547840:0:42300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df24bd lrc: 4/0,0 mode: CR/CR res: [0x200003ab1:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd40 expref: 14 pid: 42315 timeout: 1807 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.547896:0:42300:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x16:0x0].0x0 (ffff90cb83cdae00) 00010000:00010000:2.0:1769754724.547902:0:42300:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d7a00/0x11e65116e9df24f5 lrc: 3/0,1 mode: --/EX res: [0x200003ab1:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.547912:0:42300:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d7a00/0x11e65116e9df24f5 lrc: 3/0,1 mode: EX/EX res: [0x200003ab1:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.547920:0:42300:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d7a00/0x11e65116e9df24f5 lrc: 4/0,1 mode: EX/EX res: [0x200003ab1:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.547952:0:42300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769754724.548107:0:42300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb87860700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+14:2372:x1855720624214016:12345-192.168.202.6@tcp:103:ldlm_bl.0 Request processed in 307us (643us total) trans 0 rc 0/0 00000100:00100000:2.0:1769754724.548115:0:42300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 36 00010000:00010000:1.0:1769754724.548617:0:42315:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df24bd lrc: 0/0,0 mode: --/CR res: [0x200003ab1:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd40 expref: 13 pid: 42315 timeout: 1807 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1769754724.548634:0:42315:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cbaf978e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:42315:1855720644838528:192.168.202.6@tcp:104: 00010000:00010000:1.0:1769754724.548656:0:42315:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d7a00/0x11e65116e9df24f5 lrc: 3/0,1 mode: EX/EX res: [0x200003ab1:0x16:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754724.548666:0:42315:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d7a00/0x11e65116e9df24f5 lrc: 3/0,1 mode: EX/EX res: [0x200003ab1:0x16:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754724.548673:0:42315:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d7a00/0x11e65116e9df24f5 lrc: 3/0,1 mode: EX/EX res: [0x200003ab1:0x16:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1769754724.548878:0:42315:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 34359738525 00000001:00080000:1.0:1769754724.550203:0:42315:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 34359738526 00000001:00080000:1.0:1769754724.550277:0:42315:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 34359738527 00000100:00100000:1.0:1769754724.550364:0:42315:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cbaf978e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:42315:1855720644838656:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1769754724.550689:0:42315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769754724.550769:0:42315:0:(client.c:2690:ptlrpc_set_wait()) set ffff90cb9039fc00 going to sleep for 11 seconds 00000100:00100000:2.0:1769754724.551176:0:42318:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644838656 00000100:00100000:2.0:1769754724.551184:0:42318:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644838656 00000100:00100000:2.0:1769754724.551282:0:42318:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 10 00000100:00100000:2.0:1769754724.551292:0:42318:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbaf978a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:42315:x1855720644838656:12345-0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1769754724.551378:0:42318:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbaf978a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:42315:x1855720644838656:12345-0@lo:1000:mdt00_002.0 Request processed in 87us (688us total) trans 0 rc 0/0 00000100:00100000:2.0:1769754724.551386:0:42318:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 10 00000100:00100000:1.0:1769754724.551462:0:42315:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 34359738382 gen 2 00000004:00080000:1.0:1769754724.551468:0:42315:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff90cb87860000, transno 34359738382 00000100:00100000:1.0:1769754724.551490:0:42315:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cbaf978e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:42315:1855720644838656:0@lo:1000:mdt00_002.0 00000001:00080000:1.0:1769754724.551703:0:42315:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 34359738528 00000100:00100000:1.0:1769754724.551792:0:42315:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cbaf978e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:42315:1855720644838784:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1769754724.551835:0:42315:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1769754724.553673:0:42315:0:(client.c:2690:ptlrpc_set_wait()) set ffff90cb915346c0 going to sleep for 11 seconds 00000100:00100000:0.0:1769754724.554275:0:42319:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644838784 00000100:00100000:0.0:1769754724.554285:0:42319:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644838784 00000100:00100000:0.0:1769754724.554339:0:42319:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 11 00000100:00100000:0.0:1769754724.554348:0:42319:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbaf97b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:42315:x1855720644838784:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1769754724.558236:0:42319:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbaf97b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:42315:x1855720644838784:12345-0@lo:1000:mdt00_002.0 Request processed in 3879us (6390us total) trans 34359738384 rc 0/0 00000100:00100000:1.0:1769754724.558252:0:42319:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 11 00000100:00100000:2.0:1769754724.558370:0:42315:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cbaf978e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:42315:1855720644838784:0@lo:1000:mdt00_002.0 00000001:00080000:2.0:1769754724.559705:0:42315:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 34359738529 00010000:00010000:2.0:1769754724.560182:0:42315:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d7a00/0x11e65116e9df24f5 lrc: 3/0,1 mode: EX/EX res: [0x200003ab1:0x16:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.560195:0:42315:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d7a00/0x11e65116e9df24f5 lrc: 2/0,0 mode: EX/EX res: [0x200003ab1:0x16:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.560216:0:42315:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d7a00/0x11e65116e9df24f5 lrc: 3/0,0 mode: EX/EX res: [0x200003ab1:0x16:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.560223:0:42315:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff90ca887d7a00 already unused, calling callback (ffffffffc2101550) 00010000:00010000:2.0:1769754724.560228:0:42315:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d7a00/0x11e65116e9df24f5 lrc: 3/0,0 mode: EX/EX res: [0x200003ab1:0x16:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.560237:0:42315:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d7a00/0x11e65116e9df24f5 lrc: 4/0,0 mode: EX/EX res: [0x200003ab1:0x16:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.560252:0:42315:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x16:0x0].0x0 (ffff90cb83cdae00) 00010000:00010000:2.0:1769754724.560256:0:42315:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1769754724.560260:0:42315:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d7a00/0x11e65116e9df24f5 lrc: 2/0,0 mode: --/EX res: [0x200003ab1:0x16:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.560267:0:42315:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca887d7a00/0x11e65116e9df24f5 lrc: 0/0,0 mode: --/EX res: [0x200003ab1:0x16:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.560311:0:42315:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c37a00/0x11e65116e9df243f lrc: 3/0,1 mode: EX/EX res: [0x200003ab1:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000020 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.560318:0:42315:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c37a00/0x11e65116e9df243f lrc: 2/0,0 mode: EX/EX res: [0x200003ab1:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000020 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.560324:0:42315:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c37a00/0x11e65116e9df243f lrc: 3/0,0 mode: EX/EX res: [0x200003ab1:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.560329:0:42315:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff90ca82c37a00 already unused, calling callback (ffffffffc2101550) 00010000:00010000:2.0:1769754724.560332:0:42315:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c37a00/0x11e65116e9df243f lrc: 3/0,0 mode: EX/EX res: [0x200003ab1:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.560349:0:42315:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c37a00/0x11e65116e9df243f lrc: 4/0,0 mode: EX/EX res: [0x200003ab1:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000020 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.560358:0:42315:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x14:0x0].0x0 (ffff90cb87176500) 00010000:00010000:2.0:1769754724.560361:0:42315:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1769754724.560368:0:42315:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36600/0x11e65116e9df2446 lrc: 2/0,0 mode: --/CR res: [0x200003ab1:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd1d expref: 17 pid: 42313 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.560379:0:42315:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36600/0x11e65116e9df2446 lrc: 2/0,0 mode: CR/CR res: [0x200003ab1:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd1d expref: 17 pid: 42313 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.560477:0:42315:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36600/0x11e65116e9df2446 lrc: 3/0,0 mode: CR/CR res: [0x200003ab1:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd1d expref: 17 pid: 42313 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.560555:0:42315:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c36600/0x11e65116e9df2446 lrc: 3/0,0 mode: CR/CR res: [0x200003ab1:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd1d expref: 17 pid: 42313 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1769754724.560572:0:42315:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cbbbf84a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:42315:1855720644838912:192.168.202.6@tcp:105: 00000100:00100000:2.0:1769754724.560682:0:42315:0:(client.c:2690:ptlrpc_set_wait()) set ffff90cb98bcab40 going to sleep for 11 seconds 00000100:00100000:2.0:1769754724.567106:0:42315:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cbbbf84a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:42315:1855720644838912:192.168.202.6@tcp:105: 00010000:00010000:2.0:1769754724.567139:0:42315:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c37a00/0x11e65116e9df243f lrc: 2/0,0 mode: --/EX res: [0x200003ab1:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000020 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.567151:0:42315:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82c37a00/0x11e65116e9df243f lrc: 0/0,0 mode: --/EX res: [0x200003ab1:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000020 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.567191:0:42315:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df24fc lrc: 3/0,1 mode: --/EX res: [0x200003ab1:0x14:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.567220:0:42315:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df24fc lrc: 3/0,1 mode: EX/EX res: [0x200003ab1:0x14:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.567233:0:42315:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df24fc lrc: 3/0,1 mode: EX/EX res: [0x200003ab1:0x14:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.567244:0:42315:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df24fc lrc: 3/0,1 mode: EX/EX res: [0x200003ab1:0x14:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.567251:0:42315:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df24fc lrc: 2/0,0 mode: EX/EX res: [0x200003ab1:0x14:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.567258:0:42315:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df24fc lrc: 3/0,0 mode: EX/EX res: [0x200003ab1:0x14:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.567264:0:42315:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff90cb92101400 already unused, calling callback (ffffffffc2101550) 00010000:00010000:2.0:1769754724.567269:0:42315:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df24fc lrc: 3/0,0 mode: EX/EX res: [0x200003ab1:0x14:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.567277:0:42315:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df24fc lrc: 4/0,0 mode: EX/EX res: [0x200003ab1:0x14:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.567288:0:42315:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x14:0x0].0x0 (ffff90cb87176500) 00010000:00010000:2.0:1769754724.567291:0:42315:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769754724.567296:0:42315:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df24fc lrc: 2/0,0 mode: --/EX res: [0x200003ab1:0x14:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.567302:0:42315:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df24fc lrc: 0/0,0 mode: --/EX res: [0x200003ab1:0x14:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1769754724.567690:0:42315:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb87189180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+17:49739:x1855720624213888:12345-192.168.202.6@tcp:57:lhsmtool_posix.0 Request processed in 25367us (26254us total) trans 34359738530 rc 0/0 00000100:00100000:2.0:1769754724.567707:0:42315:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 198 00000020:00080000:2.0:1769754724.567714:0:42315:0:(tgt_main.c:782:tgt_ses_key_exit()) total 6 transactions per RPC 00000040:00080000:0.0:1769754724.567976:0:42347:0:(llog.c:797:llog_process_thread()) index: 8, lh_last_idx: 8 synced_idx: 8 lgh_last_idx: 8 00000040:00080000:0.0:1769754724.567981:0:42347:0:(llog_cat.c:828:llog_cat_process_common()) processing log [0x1:0x32:0x0] at index 8 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1769754724.568028:0:42347:0:(llog.c:854:llog_process_thread()) stop processing plain [0x1:0x32:0x0] index 5 count 5 00000040:00080000:0.0:1769754724.568034:0:42347:0:(llog.c:854:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 9 count 2 00000100:00100000:1.0:1769754724.573716:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:0.0:1769754724.573908:0:42317:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624214144 00000100:00100000:0.0:1769754724.573920:0:42317:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624214144 00000100:00100000:0.0:1769754724.573992:0:42317:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 65 00000100:00100000:0.0:1769754724.574013:0:42317:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbfcb5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+17:49738:x1855720624214144:12345-192.168.202.6@tcp:49:lctl.0 00000100:00100000:0.0:1769754724.574171:0:42317:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbfcb5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+17:49738:x1855720624214144:12345-192.168.202.6@tcp:49:lctl.0 Request processed in 168us (462us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754724.574178:0:42317:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 65 00000100:00100000:1.0:1769754724.582892:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:0.0:1769754724.583344:0:42316:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624214400 00000100:00100000:0.0:1769754724.583356:0:42316:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624214400 00000100:00100000:0.0:1769754724.583430:0:42316:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 66 00000100:00100000:0.0:1769754724.583441:0:42316:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbfcb6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+17:49739:x1855720624214400:12345-192.168.202.6@tcp:35:lhsmtool_posix.0 00000004:00080000:0.0:1769754724.583946:0:42316:0:(mdd_object.c:3993:mdd_close()) Object [0x200003ab1:0x16:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:0.0:1769754724.584036:0:42316:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 34359738531 00010000:00010000:0.0:1769754724.584081:0:42316:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81129600/0x11e65116e9df2503 lrc: 3/0,1 mode: --/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 42316 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754724.584097:0:42316:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81129600/0x11e65116e9df2503 lrc: 3/0,1 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 42316 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754724.584109:0:42316:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81129600/0x11e65116e9df2503 lrc: 3/0,1 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 42316 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754724.584119:0:42316:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81129600/0x11e65116e9df2503 lrc: 4/0,1 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 42316 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754724.584125:0:42316:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81129600/0x11e65116e9df2503 lrc: 4/0,1 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 42316 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754724.584131:0:42316:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81129600/0x11e65116e9df2503 lrc: 3/0,0 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 42316 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754724.584159:0:42316:0:(ldlm_lockd.c:2188:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1769754724.584276:0:42316:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbfcb6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+19:49739:x1855720624214400:12345-192.168.202.6@tcp:35:lhsmtool_posix.0 Request processed in 836us (1399us total) trans 34359738532 rc 0/0 00000100:00100000:0.0:1769754724.584285:0:42316:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 66 00000020:00080000:0.0:1769754724.584291:0:42316:0:(tgt_main.c:782:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1769754724.585285:0:42302:0:(ldlm_lockd.c:2793:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1769754724.585298:0:42302:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81129600/0x11e65116e9df2503 lrc: 2/0,0 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 42316 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.585307:0:42302:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff90ca81129600 already unused, calling callback (ffffffffc16a4360) 00010000:00010000:3.0:1769754724.585312:0:42302:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81129600/0x11e65116e9df2503 lrc: 2/0,0 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 42316 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.585336:0:42302:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81129600/0x11e65116e9df2503 lrc: 3/0,0 mode: PW/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 42316 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.585352:0:42302:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x16:0x0].0x0 (ffff90cb83c8fe00) 00010000:00010000:3.0:1769754724.585355:0:42302:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1769754724.585360:0:42302:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81129600/0x11e65116e9df2503 lrc: 1/0,0 mode: --/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 42316 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.585366:0:42302:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81129600/0x11e65116e9df2503 lrc: 0/0,0 mode: --/PW res: [0x200003ab1:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 42316 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754724.585387:0:42302:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1769754724.585401:0:42302:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1769754724.585403:0:42302:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769754724.592570:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:2.0:1769754724.592702:0:42315:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624214656 00000100:00100000:2.0:1769754724.592714:0:42315:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624214656 00000100:00100000:2.0:1769754724.592768:0:42315:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 199 00000100:00100000:2.0:1769754724.592792:0:42315:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb87860000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+19:49738:x1855720624214656:12345-192.168.202.6@tcp:58:lctl.0 00000040:00080000:2.0:1769754724.593077:0:42315:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x32:0x0].5, 136 off8872 00000100:00100000:2.0:1769754724.593159:0:42315:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb87860000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+19:49738:x1855720624214656:12345-192.168.202.6@tcp:58:lctl.0 Request processed in 382us (596us total) trans 0 rc 0/0 00000100:00100000:2.0:1769754724.593167:0:42315:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 199 00000100:00100000:2.0:1769754724.711740:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:0.0:1769754724.711839:0:42314:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624214784 00000100:00100000:0.0:1769754724.711852:0:42314:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624214784 00000100:00100000:0.0:1769754724.711918:0:42314:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 200 00000100:00100000:0.0:1769754724.711928:0:42314:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbbf84a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+19:49238:x1855720624214784:12345-192.168.202.6@tcp:60:lhsmtool_posix.0 00000100:00100000:0.0:1769754724.712047:0:42314:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbbf84a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+19:49238:x1855720624214784:12345-192.168.202.6@tcp:60:lhsmtool_posix.0 Request processed in 119us (317us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754724.712054:0:42314:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 200 00000100:00100000:1.0:1769754724.720480:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:0.0:1769754724.721085:0:42317:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624215040 00000100:00100000:0.0:1769754724.721097:0:42317:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624215040 00000100:00100000:0.0:1769754724.721158:0:42317:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 67 00000100:00100000:0.0:1769754724.721168:0:42317:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbfcb4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+19:49238:x1855720624215040:12345-192.168.202.6@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1769754724.721419:0:42317:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbfcb4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+20:49238:x1855720624215040:12345-192.168.202.6@tcp:35:lhsmtool_posix.0 Request processed in 253us (949us total) trans 34359738533 rc 0/0 00000100:00100000:0.0:1769754724.721427:0:42317:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 67 00000100:00100000:3.0:1769754724.990141:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:2.0:1769754724.992685:0:42315:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624215168 00000100:00100000:2.0:1769754724.992702:0:42315:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624215168 00000100:00100000:2.0:1769754724.992802:0:42315:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 201 00000100:00100000:2.0:1769754724.992813:0:42315:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb87861500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+20:49756:x1855720624215168:12345-192.168.202.6@tcp:101:lsof.0 00010000:00010000:2.0:1769754724.992838:0:42315:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769754724.992858:0:42315:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100200/0x11e65116e9df250a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x2b94eb038208fd5c expref: -99 pid: 42315 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.992934:0:42315:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100a00/0x11e65116e9df2511 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.992948:0:42315:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100a00/0x11e65116e9df2511 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.992959:0:42315:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100a00/0x11e65116e9df2511 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1769754724.993037:0:42315:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 13824 00000004:00200000:2.0:1769754724.993042:0:42315:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 30 stripe count 1 MDT count 1 00000004:00010000:2.0:1769754724.993070:0:42315:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100a00/0x11e65116e9df2511 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 42315 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.993093:0:42315:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100200/0x11e65116e9df250a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd5c expref: 22 pid: 42315 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.993113:0:42315:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100a00/0x11e65116e9df2511 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd5c expref: 21 pid: 42315 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754724.993121:0:42315:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff90cb83cdae00) 00010000:00010000:2.0:1769754724.993124:0:42315:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1769754724.993127:0:42315:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff90cb92100a00, rc 0) 00000100:00100000:2.0:1769754724.993202:0:42315:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb87861500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+21:49756:x1855720624215168:12345-192.168.202.6@tcp:101:lsof.0 Request processed in 391us (3070us total) trans 0 rc 0/0 00000100:00100000:2.0:1769754724.993210:0:42315:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 201 00000100:00100000:2.0:1769754725.002201:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:0.0:1769754725.002311:0:42314:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624215296 00000100:00100000:0.0:1769754725.002323:0:42314:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624215296 00000100:00100000:0.0:1769754725.002414:0:42314:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 202 00000100:00100000:0.0:1769754725.002430:0:42314:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbbf86a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:99ce4681-381f-407c-90f1-2ef98266ced9+5:49756:x1855720624215296:12345-192.168.202.6@tcp:101:lsof.0 00010000:00010000:0.0:1769754725.002457:0:42314:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769754725.002478:0:42314:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81129200/0x11e65116e9df2518 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x2b94eb038208fd63 expref: -99 pid: 42314 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754725.002589:0:42314:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81129600/0x11e65116e9df251f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754725.002604:0:42314:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81129600/0x11e65116e9df251f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754725.002617:0:42314:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81129600/0x11e65116e9df251f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1769754725.002690:0:42314:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 13824 00000004:00200000:0.0:1769754725.002694:0:42314:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 30 stripe count 1 MDT count 1 00000004:00010000:0.0:1769754725.002718:0:42314:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81129600/0x11e65116e9df251f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754725.002765:0:42314:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81129200/0x11e65116e9df2518 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd63 expref: 7 pid: 42314 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754725.002807:0:42314:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81129600/0x11e65116e9df251f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd63 expref: 6 pid: 42314 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754725.002816:0:42314:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff90cb83cdae00) 00010000:00010000:0.0:1769754725.002819:0:42314:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1769754725.002822:0:42314:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff90ca81129600, rc 0) 00000100:00100000:0.0:1769754725.002890:0:42314:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbbf86a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:99ce4681-381f-407c-90f1-2ef98266ced9+6:49756:x1855720624215296:12345-192.168.202.6@tcp:101:lsof.0 Request processed in 466us (701us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754725.002898:0:42314:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 202 00000100:00080000:1.0:1769754725.004989:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1769754725.077587:0:43453:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769754725.524234:0:3319:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb87188380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3319:1855720644839040:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1769754725.524304:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769754725.524601:0:42940:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644839040 00000100:00100000:0.0:1769754725.524610:0:42940:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644839040 00000100:00100000:0.0:1769754725.524668:0:42940:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 39 00000100:00100000:0.0:1769754725.524677:0:42940:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbfcb5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3319:x1855720644839040:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1769754725.524698:0:42940:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 49348608 pending 0 free 3774873600 avail 3771727872 00000100:00100000:0.0:1769754725.524745:0:42940:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbfcb5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3319:x1855720644839040:12345-0@lo:13:osp-pre-1-0.0 Request processed in 69us (441us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754725.524752:0:42940:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 39 00000100:00080000:1.0:1769754725.524811:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1769754725.524903:0:3319:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb87188380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3319:1855720644839040:0@lo:13:osp-pre-1-0.0 00000040:00080000:0.0:1769754725.587894:0:42347:0:(llog.c:797:llog_process_thread()) index: 8, lh_last_idx: 8 synced_idx: 8 lgh_last_idx: 8 00000040:00080000:0.0:1769754725.587903:0:42347:0:(llog_cat.c:828:llog_cat_process_common()) processing log [0x1:0x32:0x0] at index 8 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1769754725.587939:0:42347:0:(llog.c:854:llog_process_thread()) stop processing plain [0x1:0x32:0x0] index 6 count 6 00000040:00080000:0.0:1769754725.587945:0:42347:0:(llog.c:854:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 9 count 2 00000100:00100000:1.0:1769754726.036049:0:3321:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1769754726.036073:0:3321:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb8718bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3321:1855720644839168:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1769754726.036156:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769754726.036412:0:42939:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644839168 00000100:00100000:2.0:1769754726.036421:0:42939:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644839168 00000100:00100000:2.0:1769754726.036489:0:42939:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 40 00000100:00100000:2.0:1769754726.036517:0:42939:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbfcb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3321:x1855720644839168:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1769754726.036550:0:42939:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 45940736 pending 0 free 3694133248 avail 3692036096 00000100:00100000:2.0:1769754726.036592:0:42939:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbfcb4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3321:x1855720644839168:12345-0@lo:13:osp-pre-0-0.0 Request processed in 77us (436us total) trans 0 rc 0/0 00000100:00100000:2.0:1769754726.036599:0:42939:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 40 00000100:00080000:1.0:1769754726.036670:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769754726.036704:0:3321:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb8718bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3321:1855720644839168:0@lo:13:osp-pre-0-0.0 00010000:00010000:3.0:1769754726.100125:0:42301:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1769754727.123730:0:42302:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769754727.215285:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:1.0:1769754727.215335:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:1.0:1769754727.215360:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:1.0:1769754727.215378:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:1.0:1769754727.215393:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:2.0:1769754727.215536:0:42307:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624215424 00000100:00100000:2.0:1769754727.215547:0:42307:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624215424 00000100:00100000:2.0:1769754727.215614:0:42307:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 148 00000100:00100000:2.0:1769754727.215624:0:42307:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbb9559850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:cd1d8275-6a1a-49a3-b443-af2f7055b864+12:2370:x1855720624215424:12345-192.168.202.6@tcp:400:kworker.0 00000100:00100000:2.0:1769754727.215714:0:42307:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbb9559850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:cd1d8275-6a1a-49a3-b443-af2f7055b864+12:2370:x1855720624215424:12345-192.168.202.6@tcp:400:kworker.0 Request processed in 91us (443us total) trans 0 rc 0/0 00000100:00100000:2.0:1769754727.215723:0:42307:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 148 00000100:00100000:0.0:1769754727.216056:0:42938:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624215680 00000100:00100000:0.0:1769754727.216062:0:42938:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624215680 00000100:00100000:0.0:1769754727.216110:0:42938:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624215808 00000100:00100000:0.0:1769754727.216132:0:42938:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624215808 00000100:00100000:0.0:1769754727.216158:0:42938:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 69 00000100:00100000:0.0:1769754727.216165:0:42938:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbfcb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+5:2370:x1855720624215680:12345-192.168.202.6@tcp:400:kworker.0 00000100:00100000:0.0:1769754727.216202:0:42938:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbfcb5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+5:2370:x1855720624215680:12345-192.168.202.6@tcp:400:kworker.0 Request processed in 37us (865us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754727.216208:0:42938:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 69 00000100:00100000:0.0:1769754727.216219:0:42938:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 70 00000100:00100000:0.0:1769754727.216225:0:42938:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbfcb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+7:2370:x1855720624215808:12345-192.168.202.6@tcp:400:kworker.0 00000100:00100000:0.0:1769754727.216249:0:42938:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbfcb7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+7:2370:x1855720624215808:12345-192.168.202.6@tcp:400:kworker.0 Request processed in 25us (854us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754727.216255:0:42938:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 70 00000100:00100000:0.0:1769754727.216274:0:42314:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624215936 00000100:00100000:0.0:1769754727.216277:0:42314:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624215936 00000100:00100000:0.0:1769754727.216547:0:42314:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624215552 00000100:00100000:0.0:1769754727.216552:0:42314:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624215552 00000100:00100000:0.0:1769754727.216593:0:42314:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 89 00000100:00100000:0.0:1769754727.216599:0:42314:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbfcb4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:99ce4681-381f-407c-90f1-2ef98266ced9+6:2370:x1855720624215936:12345-192.168.202.6@tcp:400:kworker.0 00000100:00100000:0.0:1769754727.216630:0:42314:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbfcb4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:99ce4681-381f-407c-90f1-2ef98266ced9+6:2370:x1855720624215936:12345-192.168.202.6@tcp:400:kworker.0 Request processed in 32us (1269us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754727.216638:0:42314:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 89 00000100:00100000:0.0:1769754727.216649:0:42314:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 90 00000100:00100000:0.0:1769754727.216655:0:42314:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbfcb7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+21:2370:x1855720624215552:12345-192.168.202.6@tcp:400:kworker.0 00000100:00100000:0.0:1769754727.216677:0:42314:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbfcb7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+21:2370:x1855720624215552:12345-192.168.202.6@tcp:400:kworker.0 Request processed in 23us (1298us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754727.216683:0:42314:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 90 00000100:00080000:1.0:1769754727.216819:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1769754727.216824:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1769754727.216826:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1769754728.147928:0:43453:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769754729.108254:0:3320:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb87860e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3320:1855720644839680:0@lo:400:kworker.0 00000100:00100000:3.0:1769754729.108354:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769754729.108429:0:3320:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1769754729.108436:0:3320:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb87860a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:12a461d6-d4b3-45fe-9a31-84fa5ea63cfd:3320:1855720644839296:0@lo:400:kworker.0 00000100:00100000:3.0:1769754729.108469:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769754729.108519:0:3320:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb87861180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3320:1855720644839808:0@lo:400:kworker.0 00000100:00100000:3.0:1769754729.108545:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769754729.108579:0:3319:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb87863480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3319:1855720644839552:0@lo:400:kworker.0 00000100:00100000:3.0:1769754729.108630:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769754729.108652:0:3319:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1769754729.108657:0:3319:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb87863100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3319:1855720644839424:0@lo:400:kworker.0 00000100:00100000:3.0:1769754729.108681:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769754729.108711:0:3319:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb87860380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3319:1855720644839936:0@lo:400:kworker.0 00000100:00100000:3.0:1769754729.108732:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769754729.108778:0:42313:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644839680 00000100:00100000:3.0:1769754729.108788:0:42313:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644839680 00000100:00100000:3.0:1769754729.108847:0:42313:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644839808 00000100:00100000:3.0:1769754729.108851:0:42313:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644839808 00000100:00100000:3.0:1769754729.108873:0:42313:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644839936 00000100:00100000:3.0:1769754729.108876:0:42313:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644839936 00000100:00100000:0.0:1769754729.109852:0:42315:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 91 00000100:00100000:0.0:1769754729.109873:0:42315:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb87863b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1855720644839680:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769754729.110448:0:42315:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb87863b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1855720644839680:12345-0@lo:400:kworker.0 Request processed in 581us (2091us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754729.110460:0:42315:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 91 00000100:00100000:0.0:1769754729.110530:0:42315:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 92 00000100:00100000:0.0:1769754729.110540:0:42315:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb87863800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1855720644839808:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769754729.110573:0:42315:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb87863800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3320:x1855720644839808:12345-0@lo:400:kworker.0 Request processed in 36us (2027us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754729.110579:0:42315:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 92 00000100:00100000:0.0:1769754729.110613:0:42937:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644839552 00000100:00100000:0.0:1769754729.110623:0:42937:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644839552 00000100:00100000:0.0:1769754729.110666:0:42937:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644839424 00000100:00100000:0.0:1769754729.110670:0:42937:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644839424 00000100:00100000:0.0:1769754729.110708:0:42937:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 71 00000100:00100000:0.0:1769754729.110715:0:42937:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb87862680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3319:x1855720644839552:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769754729.110771:0:42937:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb87862680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3319:x1855720644839552:12345-0@lo:400:kworker.0 Request processed in 56us (2139us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754729.110777:0:42937:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 71 00000100:00100000:0.0:1769754729.110792:0:42937:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 72 00000100:00100000:0.0:1769754729.110798:0:42937:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb87862a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3319:x1855720644839424:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769754729.110828:0:42937:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb87862a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3319:x1855720644839424:12345-0@lo:400:kworker.0 Request processed in 31us (2147us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754729.110834:0:42937:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 72 00000100:00100000:0.0:1769754729.110913:0:3319:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb87863100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3319:1855720644839424:0@lo:400:kworker.0 00000100:00100000:0.0:1769754729.110929:0:3319:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb87863480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3319:1855720644839552:0@lo:400:kworker.0 00000100:00100000:1.0:1769754729.113536:0:42308:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644839296 00000100:00100000:1.0:1769754729.113554:0:42308:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644839296 00000100:00100000:1.0:1769754729.113634:0:42308:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 149 00000100:00100000:1.0:1769754729.113659:0:42308:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbb955b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:12a461d6-d4b3-45fe-9a31-84fa5ea63cfd+18:3320:x1855720644839296:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769754729.113787:0:42308:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbb955b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:12a461d6-d4b3-45fe-9a31-84fa5ea63cfd+18:3320:x1855720644839296:12345-0@lo:400:kworker.0 Request processed in 130us (5316us total) trans 0 rc 0/0 00000100:00100000:1.0:1769754729.113797:0:42308:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 149 00000100:00080000:1.0:1769754729.114045:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1769754729.114051:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1769754729.114055:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1769754729.123795:0:42313:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 93 00000100:00100000:3.0:1769754729.123817:0:42313:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb87861c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3319:x1855720644839936:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769754729.123976:0:42313:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb87861c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3319:x1855720644839936:12345-0@lo:400:kworker.0 Request processed in 167us (15240us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754729.123986:0:42313:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 93 00000100:00100000:0.0:1769754729.124204:0:3319:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb87860380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3319:1855720644839936:0@lo:400:kworker.0 00000100:00100000:2.0:1769754729.125172:0:3320:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb87860e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3320:1855720644839680:0@lo:400:kworker.0 00000100:00100000:2.0:1769754729.125200:0:3320:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb87860a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:12a461d6-d4b3-45fe-9a31-84fa5ea63cfd:3320:1855720644839296:0@lo:400:kworker.0 00000100:00100000:2.0:1769754729.125213:0:3320:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb87861180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3320:1855720644839808:0@lo:400:kworker.0 00010000:00010000:3.0:1769754729.172620:0:42301:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1769754729.710619:0:7447:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 34359738384 is committed 00000001:00080000:0.0:1769754729.711437:0:7447:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0001: transno 34359738383 is committed 00000001:00080000:2.0:1769754729.777547:0:5807:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 34359738533 is committed 00000020:00080000:2.0:1769754729.777585:0:5807:0:(update_trans.c:60:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff90cb88bc49c0 refcount 2 committed 0 result 0 batchid 30064771788 00000020:00080000:2.0:1769754729.777591:0:5807:0:(update_trans.c:69:top_multiple_thandle_dump()) st ffff90cb9135bb00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff90cb88bc4840 00000020:00080000:2.0:1769754729.777595:0:5807:0:(update_trans.c:69:top_multiple_thandle_dump()) st ffff90cb9135be00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff90ca8cc03400 00000020:00080000:2.0:1769754729.778531:0:42352:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 30064771788 committed batchid 30064771787 00000020:00080000:2.0:1769754729.781186:0:42352:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 30064771787 00010000:00010000:1.0:1769754730.203242:0:42302:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769754730.644754:0:3319:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb87861180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3319:1855720644840064:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1769754730.644855:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769754730.645373:0:42940:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644840064 00000100:00100000:3.0:1769754730.645385:0:42940:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644840064 00000100:00100000:3.0:1769754730.645440:0:42940:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 41 00000100:00100000:3.0:1769754730.645450:0:42940:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb87860a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1855720644840064:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1769754730.645518:0:42940:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 49348608 pending 0 free 3774873600 avail 3772776448 00000100:00100000:3.0:1769754730.645656:0:42940:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb87860a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1855720644840064:12345-0@lo:13:osp-pre-1-0.0 Request processed in 207us (801us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754730.646823:0:42940:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 41 00000100:00100000:3.0:1769754730.647093:0:3319:0:(client.c:3081:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 34359738384 gen 2 00000004:00080000:3.0:1769754730.647101:0:3319:0:(osp_trans.c:1035:osp_request_commit_cb()) trans no 34359738384 committed transno 34359738384 00000100:00100000:3.0:1769754730.647144:0:3319:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb87861180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3319:1855720644840064:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1769754731.155908:0:3321:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1769754731.155935:0:3321:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb8718b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3321:1855720644840192:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1769754731.156004:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769754731.156274:0:42940:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644840192 00000100:00100000:3.0:1769754731.156283:0:42940:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644840192 00000100:00100000:3.0:1769754731.156359:0:42940:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 42 00000100:00100000:3.0:1769754731.156368:0:42940:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbfcb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3321:x1855720644840192:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1769754731.156409:0:42940:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 45940736 pending 0 free 3694133248 avail 3692036096 00000100:00100000:3.0:1769754731.156461:0:42940:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbfcb4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3321:x1855720644840192:12345-0@lo:13:osp-pre-0-0.0 Request processed in 93us (456us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754731.156468:0:42940:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 42 00000100:00080000:1.0:1769754731.156734:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769754731.157037:0:3321:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb8718b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3321:1855720644840192:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1769754731.219777:0:43453:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1769754732.243666:0:42301:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769754732.334101:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:3.0:1769754732.334160:0:42307:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624216064 00000100:00100000:3.0:1769754732.334172:0:42307:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624216064 00000100:00100000:3.0:1769754732.334291:0:42307:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 150 00000100:00100000:3.0:1769754732.334303:0:42307:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbb9558450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:cd1d8275-6a1a-49a3-b443-af2f7055b864+12:2371:x1855720624216064:12345-192.168.202.6@tcp:400:kworker.0 00000100:00100000:3.0:1769754732.334392:0:42307:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbb9558450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:cd1d8275-6a1a-49a3-b443-af2f7055b864+12:2371:x1855720624216064:12345-192.168.202.6@tcp:400:kworker.0 Request processed in 92us (301us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754732.334400:0:42307:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 150 00000100:00100000:1.0:1769754732.340639:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:1.0:1769754732.340699:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:1.0:1769754732.340734:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:2.0:1769754732.340765:0:42314:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624216576 00000100:00100000:1.0:1769754732.340771:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:2.0:1769754732.340777:0:42314:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624216576 00000100:00100000:2.0:1769754732.340831:0:42314:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624216192 00000100:00100000:2.0:1769754732.340834:0:42314:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624216192 00000100:00100000:3.0:1769754732.340847:0:42313:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 94 00000100:00100000:2.0:1769754732.340855:0:42314:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 95 00000100:00100000:3.0:1769754732.340863:0:42313:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbfcb4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:99ce4681-381f-407c-90f1-2ef98266ced9+6:2371:x1855720624216576:12345-192.168.202.6@tcp:400:kworker.0 00000100:00100000:2.0:1769754732.340867:0:42314:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbfcb7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+12:2371:x1855720624216192:12345-192.168.202.6@tcp:400:kworker.0 00000100:00100000:2.0:1769754732.340936:0:42314:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbfcb7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+12:2371:x1855720624216192:12345-192.168.202.6@tcp:400:kworker.0 Request processed in 73us (200us total) trans 0 rc 0/0 00000100:00100000:2.0:1769754732.340943:0:42314:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 95 00000100:00100000:3.0:1769754732.340982:0:42313:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbfcb4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:99ce4681-381f-407c-90f1-2ef98266ced9+6:2371:x1855720624216576:12345-192.168.202.6@tcp:400:kworker.0 Request processed in 124us (280us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754732.340993:0:42313:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 94 00000100:00080000:2.0:1769754732.341064:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1769754732.341214:0:42938:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624216320 00000100:00100000:0.0:1769754732.341220:0:42938:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624216320 00000100:00080000:2.0:1769754732.341268:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1769754732.341272:0:42938:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624216448 00000100:00100000:0.0:1769754732.341277:0:42938:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624216448 00000100:00080000:2.0:1769754732.341297:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1769754732.341308:0:42938:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 73 00000100:00100000:0.0:1769754732.341316:0:42938:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbfcb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+5:2371:x1855720624216320:12345-192.168.202.6@tcp:400:kworker.0 00000100:00100000:0.0:1769754732.341353:0:42938:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbfcb5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+5:2371:x1855720624216320:12345-192.168.202.6@tcp:400:kworker.0 Request processed in 38us (723us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754732.341360:0:42938:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 73 00000100:00100000:0.0:1769754732.341376:0:42938:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 74 00000100:00100000:0.0:1769754732.341383:0:42938:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbfcb7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+6:2371:x1855720624216448:12345-192.168.202.6@tcp:400:kworker.0 00000100:00100000:0.0:1769754732.341416:0:42938:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbfcb7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+6:2371:x1855720624216448:12345-192.168.202.6@tcp:400:kworker.0 Request processed in 33us (647us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754732.341422:0:42938:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 74 00010000:00010000:1.0:1769754733.275414:0:42302:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769754734.228133:0:3321:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cbbbf84700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:12a461d6-d4b3-45fe-9a31-84fa5ea63cfd:3321:1855720644840320:0@lo:400:kworker.0 00000100:00100000:2.0:1769754734.228205:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769754734.228259:0:3321:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cbbd7e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3321:1855720644840832:0@lo:400:kworker.0 00000100:00100000:2.0:1769754734.228287:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769754734.228314:0:3321:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1769754734.228320:0:3321:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb98352300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3321:1855720644840704:0@lo:400:kworker.0 00000100:00100000:2.0:1769754734.228340:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769754734.228368:0:3318:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cbbbf84000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3318:1855720644840448:0@lo:400:kworker.0 00000100:00100000:2.0:1769754734.228393:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769754734.228410:0:3318:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cbbd7e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3318:1855720644840960:0@lo:400:kworker.0 00000100:00100000:2.0:1769754734.228430:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769754734.228446:0:3318:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1769754734.228451:0:3318:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb98352a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3318:1855720644840576:0@lo:400:kworker.0 00000100:00100000:2.0:1769754734.228472:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769754734.228750:0:43811:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644840832 00000100:00100000:3.0:1769754734.228760:0:43811:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644840832 00000100:00100000:3.0:1769754734.228814:0:43811:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644840704 00000100:00100000:3.0:1769754734.228817:0:43811:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644840704 00000100:00100000:3.0:1769754734.228829:0:43811:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644840960 00000100:00100000:3.0:1769754734.228831:0:43811:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644840960 00000100:00100000:3.0:1769754734.228846:0:43811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 96 00000100:00100000:3.0:1769754734.228855:0:43811:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbd7e0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3321:x1855720644840832:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769754734.228911:0:43811:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbd7e0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3321:x1855720644840832:12345-0@lo:400:kworker.0 Request processed in 58us (623us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754734.228919:0:43811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 96 00000100:00100000:3.0:1769754734.228936:0:43811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 97 00000100:00100000:3.0:1769754734.228942:0:43811:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbd7e1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3321:x1855720644840704:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769754734.228967:0:43811:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbd7e1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3321:x1855720644840704:12345-0@lo:400:kworker.0 Request processed in 26us (627us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754734.228973:0:43811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 97 00000100:00100000:3.0:1769754734.228983:0:43811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 98 00000100:00100000:3.0:1769754734.228989:0:43811:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbd7e2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1855720644840960:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769754734.229128:0:42937:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644840448 00000100:00100000:2.0:1769754734.229132:0:42937:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644840448 00000100:00100000:2.0:1769754734.229160:0:42937:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644840576 00000100:00100000:2.0:1769754734.229162:0:42937:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644840576 00000100:00100000:2.0:1769754734.229181:0:42937:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 75 00000100:00100000:2.0:1769754734.229187:0:42937:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbd7e3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3318:x1855720644840448:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769754734.229232:0:43811:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbd7e2300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1855720644840960:12345-0@lo:400:kworker.0 Request processed in 242us (800us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754734.229239:0:43811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 98 00000100:00100000:3.0:1769754734.229263:0:43451:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 76 00000100:00100000:1.0:1769754734.229267:0:42307:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644840320 00000100:00100000:3.0:1769754734.229270:0:43451:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90ca81ffb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3318:x1855720644840576:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1769754734.229271:0:42307:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644840320 00000100:00100000:1.0:1769754734.229302:0:42307:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 151 00000100:00100000:1.0:1769754734.229309:0:42307:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbb955a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:12a461d6-d4b3-45fe-9a31-84fa5ea63cfd+18:3321:x1855720644840320:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769754734.229319:0:42937:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbd7e3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3318:x1855720644840448:12345-0@lo:400:kworker.0 Request processed in 132us (925us total) trans 0 rc 0/0 00000100:00100000:2.0:1769754734.229325:0:42937:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 75 00000100:00100000:1.0:1769754734.229339:0:42307:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbb955a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:12a461d6-d4b3-45fe-9a31-84fa5ea63cfd+18:3321:x1855720644840320:12345-0@lo:400:kworker.0 Request processed in 31us (1135us total) trans 0 rc 0/0 00000100:00100000:1.0:1769754734.229345:0:42307:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 151 00000100:00100000:3.0:1769754734.229391:0:43451:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90ca81ffb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3318:x1855720644840576:12345-0@lo:400:kworker.0 Request processed in 122us (917us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754734.229398:0:43451:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 76 00000100:00080000:3.0:1769754734.229419:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1769754734.229428:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1769754734.229431:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1769754734.229465:0:3318:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb98352a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3318:1855720644840576:0@lo:400:kworker.0 00000100:00100000:3.0:1769754734.229476:0:3318:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cbbbf84000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3318:1855720644840448:0@lo:400:kworker.0 00000100:00100000:3.0:1769754734.229484:0:3318:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cbbd7e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3318:1855720644840960:0@lo:400:kworker.0 00000100:00100000:2.0:1769754734.229639:0:3321:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb98352300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3321:1855720644840704:0@lo:400:kworker.0 00000100:00100000:2.0:1769754734.229814:0:3321:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cbbbf84700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:12a461d6-d4b3-45fe-9a31-84fa5ea63cfd:3321:1855720644840320:0@lo:400:kworker.0 00000100:00100000:2.0:1769754734.229825:0:3321:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cbbd7e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3321:1855720644840832:0@lo:400:kworker.0 00010000:00010000:2.0:1769754734.291763:0:43453:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00080000:2.0:1769754734.795313:0:5807:0:(update_trans.c:1333:distribute_txn_batchid_cb()) lustre-MDT0000: 30064771788 batchid updated 00000020:00080000:0.0:1769754734.795382:0:42352:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 30064771788 00000020:00080000:0.0:1769754734.795395:0:42352:0:(update_trans.c:1571:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 30064771788 00010000:00010000:2.0:1769754735.316331:0:42301:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769754735.763992:0:3318:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1769754735.764038:0:3318:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb8718b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3318:1855720644841088:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1769754735.764132:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769754735.764203:0:42939:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644841088 00000100:00100000:3.0:1769754735.764221:0:42939:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644841088 00000100:00100000:3.0:1769754735.764297:0:42939:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 43 00000100:00100000:3.0:1769754735.764309:0:42939:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb8718a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3318:x1855720644841088:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1769754735.764353:0:42939:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 49348608 pending 0 free 3774873600 avail 3772776448 00000100:00100000:3.0:1769754735.764431:0:42939:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb8718a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3318:x1855720644841088:12345-0@lo:13:osp-pre-1-0.0 Request processed in 124us (298us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754735.764440:0:42939:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 43 00000100:00100000:0.0:1769754735.764574:0:3318:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb8718b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3318:1855720644841088:0@lo:13:osp-pre-1-0.0 00000040:00080000:3.0:1769754735.832038:0:42347:0:(llog.c:797:llog_process_thread()) index: 8, lh_last_idx: 8 synced_idx: 8 lgh_last_idx: 8 00000040:00080000:3.0:1769754735.832050:0:42347:0:(llog_cat.c:828:llog_cat_process_common()) processing log [0x1:0x32:0x0] at index 8 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1769754735.832289:0:42347:0:(llog.c:854:llog_process_thread()) stop processing plain [0x1:0x32:0x0] index 6 count 5 00000040:00080000:3.0:1769754735.832296:0:42347:0:(llog.c:854:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 9 count 2 00000100:00100000:2.0:1769754736.277790:0:3320:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb8718b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3320:1855720644841216:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1769754736.277895:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769754736.281803:0:42940:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644841216 00000100:00100000:0.0:1769754736.281826:0:42940:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644841216 00000100:00100000:0.0:1769754736.282033:0:42940:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 44 00000100:00100000:0.0:1769754736.282047:0:42940:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90ca81ff9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1855720644841216:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1769754736.282103:0:42940:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 45940736 pending 0 free 3694133248 avail 3692036096 00000100:00100000:0.0:1769754736.282236:0:42940:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90ca81ff9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3320:x1855720644841216:12345-0@lo:13:osp-pre-0-0.0 Request processed in 192us (4339us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754736.282245:0:42940:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 44 00000100:00100000:1.0:1769754736.283816:0:3320:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb8718b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3320:1855720644841216:0@lo:13:osp-pre-0-0.0 00000100:00080000:3.0:1769754736.284988:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1769754736.339945:0:42302:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1769754737.366459:0:43453:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1769754737.460914:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:2.0:1769754737.460981:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:2.0:1769754737.461012:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:2.0:1769754737.461035:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:2.0:1769754737.461064:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:2.0:1769754737.461100:0:42313:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624216832 00000100:00100000:2.0:1769754737.461110:0:42313:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624216832 00000100:00100000:2.0:1769754737.461168:0:42313:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624217216 00000100:00100000:2.0:1769754737.461171:0:42313:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624217216 00000100:00100000:2.0:1769754737.461199:0:42313:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 99 00000100:00100000:2.0:1769754737.461210:0:42313:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90ca81ff8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+12:2369:x1855720624216832:12345-192.168.202.6@tcp:400:kworker.0 00000100:00100000:2.0:1769754737.461306:0:42313:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90ca81ff8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+12:2369:x1855720624216832:12345-192.168.202.6@tcp:400:kworker.0 Request processed in 98us (293us total) trans 0 rc 0/0 00000100:00100000:2.0:1769754737.461332:0:42313:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 99 00000100:00100000:2.0:1769754737.461350:0:42313:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 100 00000100:00100000:2.0:1769754737.461356:0:42313:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbb95e6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:99ce4681-381f-407c-90f1-2ef98266ced9+6:2369:x1855720624217216:12345-192.168.202.6@tcp:400:kworker.0 00000100:00100000:2.0:1769754737.461382:0:42313:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbb95e6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:99ce4681-381f-407c-90f1-2ef98266ced9+6:2369:x1855720624217216:12345-192.168.202.6@tcp:400:kworker.0 Request processed in 27us (346us total) trans 0 rc 0/0 00000100:00100000:2.0:1769754737.461388:0:42313:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 100 00000100:00100000:2.0:1769754737.461409:0:42938:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624217088 00000100:00100000:2.0:1769754737.461412:0:42938:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624217088 00000100:00100000:2.0:1769754737.461439:0:42938:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624216960 00000100:00100000:2.0:1769754737.461443:0:42938:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624216960 00000100:00100000:2.0:1769754737.461463:0:42938:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 77 00000100:00100000:2.0:1769754737.461468:0:42938:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90ca81ff9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+6:2369:x1855720624217088:12345-192.168.202.6@tcp:400:kworker.0 00000100:00100000:2.0:1769754737.461523:0:42938:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90ca81ff9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+6:2369:x1855720624217088:12345-192.168.202.6@tcp:400:kworker.0 Request processed in 53us (541us total) trans 0 rc 0/0 00000100:00100000:2.0:1769754737.461529:0:42938:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 77 00000100:00100000:2.0:1769754737.461542:0:42938:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 56 00000100:00100000:2.0:1769754737.461548:0:42938:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbb95e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+5:2368:x1855720624216960:12345-192.168.202.6@tcp:9:kworker.0 00000020:00000080:2.0:1769754737.461568:0:42938:0:(genops.c:1485:class_disconnect()) disconnect: cookie 0x11e65116e9df2382 00000100:00080000:2.0:1769754737.461577:0:42938:0:(import.c:65:import_set_state_nolock()) ffff90ca88729800 : changing import state from FULL to CLOSED 00000001:00000004:2.0:1769754737.461615:0:42938:0:(tgt_lastrcvd.c:696:tgt_server_data_update()) lustre-OST0000_UUID: mount_count is 8, last_transno is 34359738382 00000100:00100000:2.0:1769754737.461862:0:42938:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbb95e4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+2:2368:x1855720624216960:12345-192.168.202.6@tcp:9:kworker.0 Request processed in 313us (796us total) trans 0 rc 0/0 00000100:00100000:2.0:1769754737.461870:0:42938:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 56 00000100:00080000:2.0:1769754737.461887:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1769754737.461891:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1769754737.461901:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1769754737.462057:0:42301:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769754737.462158:0:42308:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624216704 00000100:00100000:1.0:1769754737.462171:0:42308:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624216704 00000100:00100000:1.0:1769754737.462220:0:42308:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 152 00000100:00100000:1.0:1769754737.462228:0:42308:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbb955b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:cd1d8275-6a1a-49a3-b443-af2f7055b864+12:2369:x1855720624216704:12345-192.168.202.6@tcp:400:kworker.0 00000100:00100000:1.0:1769754737.462267:0:42308:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbb955b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:cd1d8275-6a1a-49a3-b443-af2f7055b864+12:2369:x1855720624216704:12345-192.168.202.6@tcp:400:kworker.0 Request processed in 40us (1367us total) trans 0 rc 0/0 00000100:00100000:1.0:1769754737.462274:0:42308:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 152 00000001:02000400:2.0:1769754737.789738:0:46154:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-pcc test 17: Test auto attach for layout refresh ========================================================== 01:32:16 (1769754736) 00010000:00010000:1.0:1769754738.390291:0:42302:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769754739.348962:0:3319:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb8718a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3319:1855720644841600:0@lo:400:kworker.0 00000100:00100000:2.0:1769754739.348990:0:3318:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb87189c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3318:1855720644841472:0@lo:400:kworker.0 00000100:00100000:3.0:1769754739.349067:0:3321:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb8718aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:12a461d6-d4b3-45fe-9a31-84fa5ea63cfd:3321:1855720644841344:0@lo:400:kworker.0 00000100:00100000:1.0:1769754739.349067:0:3320:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb87189f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3320:1855720644841728:0@lo:400:kworker.0 00000100:00100000:0.0:1769754739.353691:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769754739.353867:0:43451:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644841600 00000100:00100000:0.0:1769754739.354099:0:43451:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644841600 00000100:00100000:0.0:1769754739.354186:0:43451:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 78 00000100:00100000:0.0:1769754739.354198:0:43451:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb87189880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3319:x1855720644841600:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769754739.354272:0:43451:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb87189880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3319:x1855720644841600:12345-0@lo:400:kworker.0 Request processed in 77us (593us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754739.354280:0:43451:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 78 00000100:00080000:0.0:1769754739.354309:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1769754739.354352:0:3319:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb8718a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3319:1855720644841600:0@lo:400:kworker.0 00000100:00100000:2.0:1769754739.354798:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769754739.354851:0:3318:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb87188000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3318:1855720644841984:0@lo:400:kworker.0 00000100:00100000:2.0:1769754739.355250:0:3318:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769754739.356299:0:42937:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644841472 00000100:00100000:0.0:1769754739.356312:0:42937:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644841472 00000100:00100000:3.0:1769754739.356549:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769754739.356596:0:3321:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb8718ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3321:1855720644841856:0@lo:400:kworker.0 00000100:00100000:3.0:1769754739.356629:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769754739.356665:0:42315:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644841984 00000100:00100000:3.0:1769754739.356669:0:42315:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644841984 00000100:00100000:3.0:1769754739.356700:0:42315:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644841856 00000100:00100000:3.0:1769754739.356702:0:42315:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644841856 00000100:00100000:3.0:1769754739.356720:0:42315:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 101 00000100:00100000:3.0:1769754739.356757:0:42315:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbb95e6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1855720644841984:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769754739.356812:0:42315:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbb95e6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3318:x1855720644841984:12345-0@lo:400:kworker.0 Request processed in 57us (1920us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754739.356820:0:42315:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 101 00000100:00100000:3.0:1769754739.356836:0:42315:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 102 00000100:00100000:3.0:1769754739.356841:0:42315:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb87861180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3321:x1855720644841856:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1769754739.356864:0:42315:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb87861180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3321:x1855720644841856:12345-0@lo:400:kworker.0 Request processed in 23us (234us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754739.356870:0:42315:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 102 00000100:00100000:3.0:1769754739.356916:0:3321:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb8718ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3321:1855720644841856:0@lo:400:kworker.0 00000100:00100000:0.0:1769754739.357304:0:42937:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 79 00000100:00100000:0.0:1769754739.357316:0:42937:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbb95e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3318:x1855720644841472:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1769754739.357363:0:42937:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbb95e4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3318:x1855720644841472:12345-0@lo:400:kworker.0 Request processed in 51us (2566us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754739.357368:0:42937:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 79 00000100:00080000:2.0:1769754739.358078:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1769754739.358087:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1769754739.358110:0:42307:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644841344 00000100:00100000:2.0:1769754739.358118:0:42307:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644841344 00000100:00100000:2.0:1769754739.358170:0:42307:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 153 00000100:00100000:2.0:1769754739.358179:0:42307:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbb9558850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:12a461d6-d4b3-45fe-9a31-84fa5ea63cfd+18:3321:x1855720644841344:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769754739.358233:0:42307:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbb9558850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:12a461d6-d4b3-45fe-9a31-84fa5ea63cfd+18:3321:x1855720644841344:12345-0@lo:400:kworker.0 Request processed in 56us (1686us total) trans 0 rc 0/0 00000100:00100000:2.0:1769754739.358240:0:42307:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 153 00000100:00100000:2.0:1769754739.358304:0:3318:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb87189c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3318:1855720644841472:0@lo:400:kworker.0 00000100:00100000:2.0:1769754739.358319:0:3318:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb87188000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3318:1855720644841984:0@lo:400:kworker.0 00000100:00100000:3.0:1769754739.359898:0:3321:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb8718aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:12a461d6-d4b3-45fe-9a31-84fa5ea63cfd:3321:1855720644841344:0@lo:400:kworker.0 00000100:00100000:1.0:1769754739.360678:0:3320:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1769754739.362189:0:42313:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644841728 00000100:00100000:2.0:1769754739.362204:0:42313:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644841728 00000100:00100000:2.0:1769754739.362300:0:42313:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 103 00000100:00100000:2.0:1769754739.362312:0:42313:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbfcb6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1855720644841728:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1769754739.362422:0:42313:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbfcb6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3320:x1855720644841728:12345-0@lo:400:kworker.0 Request processed in 112us (1745us total) trans 0 rc 0/0 00000100:00100000:2.0:1769754739.362430:0:42313:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 103 00000100:00100000:1.0:1769754739.362621:0:3320:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb87189f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3320:1855720644841728:0@lo:400:kworker.0 00010000:00010000:2.0:1769754739.412113:0:43453:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1769754739.598372:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:0.0:1769754739.598449:0:43811:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624217472 00000100:00100000:0.0:1769754739.598461:0:43811:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624217472 00000100:00100000:0.0:1769754739.598571:0:43811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 203 00000100:00100000:0.0:1769754739.598649:0:43811:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb87860e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+12:50148:x1855720624217472:12345-192.168.202.6@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1769754739.598683:0:43811:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00080000:2.0:1769754739.598702:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1769754739.598710:0:43811:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1800/0x11e65116e9df2526 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x2b94eb038208fd6a expref: -99 pid: 43811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754739.598814:0:43811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0c00/0x11e65116e9df252d lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754739.598832:0:43811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0c00/0x11e65116e9df252d lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754739.598847:0:43811:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0c00/0x11e65116e9df252d lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754739.598865:0:43811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0200/0x11e65116e9df2534 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754739.600380:0:43811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0200/0x11e65116e9df2534 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754739.600398:0:43811:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0200/0x11e65116e9df2534 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754739.600688:0:43811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1e00/0x11e65116e9df253b lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754739.600699:0:43811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1e00/0x11e65116e9df253b lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754739.600729:0:43811:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1e00/0x11e65116e9df253b lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1769754739.600744:0:43811:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000002:0x2:0x0]: returning size 10752 00010000:00010000:0.0:1769754739.601365:0:43811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0c00/0x11e65116e9df252d lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754739.601373:0:43811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0c00/0x11e65116e9df252d lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754739.601378:0:43811:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0c00/0x11e65116e9df252d lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754739.601382:0:43811:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff90ca82ad0c00 already unused, calling callback (ffffffffc2101550) 00010000:00010000:0.0:1769754739.601386:0:43811:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0c00/0x11e65116e9df252d lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754739.601392:0:43811:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0c00/0x11e65116e9df252d lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754739.601402:0:43811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x0 (ffff90cb82b5ab00) 00010000:00010000:0.0:1769754739.601404:0:43811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769754739.601409:0:43811:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0c00/0x11e65116e9df252d lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754739.601412:0:43811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0c00/0x11e65116e9df252d lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754739.601426:0:43811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0200/0x11e65116e9df2534 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754739.601430:0:43811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0200/0x11e65116e9df2534 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754739.601440:0:43811:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0200/0x11e65116e9df2534 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754739.601444:0:43811:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff90ca82ad0200 already unused, calling callback (ffffffffc2101550) 00010000:00010000:0.0:1769754739.601446:0:43811:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0200/0x11e65116e9df2534 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754739.601449:0:43811:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0200/0x11e65116e9df2534 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754739.601453:0:43811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x8d45d6bc (ffff90cb83c8f300) 00010000:00010000:0.0:1769754739.601455:0:43811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769754739.601456:0:43811:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0200/0x11e65116e9df2534 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754739.601460:0:43811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0200/0x11e65116e9df2534 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754739.601850:0:43811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1800/0x11e65116e9df2526 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd6a expref: 15 pid: 43811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754739.601865:0:43811:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1e00/0x11e65116e9df253b lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd6a expref: 14 pid: 43811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754739.601880:0:43811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff90cb87176c00) 00010000:00010000:0.0:1769754739.601882:0:43811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769754739.601884:0:43811:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff90ca82ad1e00, rc 0) 00000100:00100000:0.0:1769754739.601967:0:43811:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb87860e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+14:50148:x1855720624217472:12345-192.168.202.6@tcp:101:lhsmtool_posix.0 Request processed in 3386us (3609us total) trans 34359738534 rc 0/0 00000100:00100000:0.0:1769754739.601988:0:43811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 203 00000100:00100000:1.0:1769754739.606005:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:3.0:1769754739.606329:0:42299:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624217600 00000100:00100000:3.0:1769754739.606341:0:42299:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624217600 00000100:00100000:3.0:1769754739.606418:0:42299:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 15 00000100:00100000:3.0:1769754739.606431:0:42299:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbfcb6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+14:2369:x1855720624217600:12345-192.168.202.6@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769754739.606454:0:42299:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769754739.606483:0:42299:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1e00/0x11e65116e9df253b lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd6a expref: 14 pid: 43811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754739.606742:0:42299:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff90cb87176c00) 00010000:00010000:3.0:1769754739.606750:0:42299:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1769754739.606859:0:42299:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbfcb6a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+13:2369:x1855720624217600:12345-192.168.202.6@tcp:103:ldlm_bl.0 Request processed in 430us (866us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754739.606868:0:42299:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 15 00000100:00100000:2.0:1769754739.608539:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:0.0:1769754739.608632:0:43811:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624217728 00000100:00100000:0.0:1769754739.608653:0:43811:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624217728 00000100:00100000:0.0:1769754739.608942:0:43811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 204 00000100:00100000:0.0:1769754739.608954:0:43811:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbb95e5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+13:50148:x1855720624217728:12345-192.168.202.6@tcp:59:lhsmtool_posix.0 00000100:00100000:0.0:1769754739.609108:0:43811:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbb95e5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+13:50148:x1855720624217728:12345-192.168.202.6@tcp:59:lhsmtool_posix.0 Request processed in 157us (614us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754739.609117:0:43811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 204 00000100:00100000:3.0:1769754739.864728:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:3.0:1769754739.864768:0:43811:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624217984 00000100:00100000:3.0:1769754739.864780:0:43811:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624217984 00000100:00100000:3.0:1769754739.864838:0:43811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 205 00000100:00100000:3.0:1769754739.864847:0:43811:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb87860380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+13:50168:x1855720624217984:12345-192.168.202.6@tcp:101:bash.0 00010000:00010000:3.0:1769754739.864871:0:43811:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769754739.864876:0:43811:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1769754739.864909:0:43811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100a00/0x11e65116e9df2511 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd5c expref: 13 pid: 42315 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754739.864931:0:43811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff90cb83cdae00) 00010000:00010000:3.0:1769754739.864935:0:43811:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1769754739.864950:0:43811:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3600/0x11e65116e9df2549 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x2b94eb038208fd71 expref: -99 pid: 43811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754739.865103:0:43811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3a00/0x11e65116e9df2550 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754739.865117:0:43811:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81129600/0x11e65116e9df251f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd63 expref: 5 pid: 42314 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754739.865125:0:43811:0:(ldlm_resource.c:1714:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3a00/0x11e65116e9df2550 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754739.865164:0:43811:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81129600/0x11e65116e9df251f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd63 expref: 5 pid: 42314 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754739.865180:0:43811:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81129600/0x11e65116e9df251f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd63 expref: 5 pid: 42314 timeout: 1822 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1769754739.865213:0:43811:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb87863480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:43811:1855720644842112:192.168.202.6@tcp:104: 00000100:00100000:3.0:1769754739.865312:0:43811:0:(client.c:2690:ptlrpc_set_wait()) set ffff90cba065f840 going to sleep for 11 seconds 00000100:00100000:0.0:1769754739.875613:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:3.0:1769754739.877694:0:42299:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624218112 00000100:00100000:3.0:1769754739.877713:0:42299:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624218112 00010000:00010000:3.0:1769754739.877788:0:42299:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81129600/0x11e65116e9df251f lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd63 expref: 6 pid: 42314 timeout: 1822 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1769754739.877832:0:42299:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 37 00000100:00100000:3.0:1769754739.877841:0:42299:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb8718a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:99ce4681-381f-407c-90f1-2ef98266ced9+6:2371:x1855720624218112:12345-192.168.202.6@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769754739.877859:0:42299:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769754739.877867:0:42299:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81129600/0x11e65116e9df251f lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd63 expref: 6 pid: 42314 timeout: 1822 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754739.877882:0:42299:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81129600/0x11e65116e9df251f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd63 expref: 6 pid: 42314 timeout: 1822 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754739.877910:0:42299:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff90cb83cdae00) 00010000:00010000:3.0:1769754739.877916:0:42299:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3a00/0x11e65116e9df2550 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754739.877928:0:42299:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3a00/0x11e65116e9df2550 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754739.877936:0:42299:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3a00/0x11e65116e9df2550 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754739.877959:0:42299:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1769754739.878020:0:43811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca81129600/0x11e65116e9df251f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd63 expref: 6 pid: 42314 timeout: 1822 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1769754739.878039:0:43811:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb87863480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:43811:1855720644842112:192.168.202.6@tcp:104: 00000100:00100000:3.0:1769754739.878049:0:42299:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb8718a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:99ce4681-381f-407c-90f1-2ef98266ced9+5:2371:x1855720624218112:12345-192.168.202.6@tcp:103:ldlm_bl.0 Request processed in 206us (2573us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754739.878058:0:42299:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 37 00010000:00010000:0.0:1769754739.878067:0:43811:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3a00/0x11e65116e9df2550 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754739.878078:0:43811:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3a00/0x11e65116e9df2550 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754739.878085:0:43811:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3a00/0x11e65116e9df2550 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754739.878115:0:43811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1800/0x11e65116e9df2557 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754739.878126:0:43811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1800/0x11e65116e9df2557 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754739.878145:0:43811:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1800/0x11e65116e9df2557 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1769754739.878667:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000004:00080000:0.0:1769754739.881383:0:43811:0:(osp_object.c:1657:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x106:0x0], index 0: 0 00010000:00010000:2.0:1769754739.887149:0:43811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100200/0x11e65116e9df255e lrc: 3/1,0 mode: --/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754739.887185:0:43811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100200/0x11e65116e9df255e lrc: 3/1,0 mode: PR/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754739.887200:0:43811:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100200/0x11e65116e9df255e lrc: 3/1,0 mode: PR/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754739.887261:0:43811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3a00/0x11e65116e9df2550 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754739.887269:0:43811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3a00/0x11e65116e9df2550 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754739.887277:0:43811:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3a00/0x11e65116e9df2550 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754739.887283:0:43811:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff90ca820a3a00 already unused, calling callback (ffffffffc2101550) 00010000:00010000:2.0:1769754739.887287:0:43811:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3a00/0x11e65116e9df2550 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754739.887296:0:43811:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3a00/0x11e65116e9df2550 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754739.887309:0:43811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff90cb83cdae00) 00010000:00010000:2.0:1769754739.887312:0:43811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1769754739.887318:0:43811:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3a00/0x11e65116e9df2550 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754739.887325:0:43811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3a00/0x11e65116e9df2550 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1769754739.887354:0:43811:0:(mdt_handler.c:4458:mdt_save_lock()) ### save lock request 00000000b9a2c1ea reply state 00000000e0029349 transno 34359738535 ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1800/0x11e65116e9df2557 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1769754739.887364:0:43811:0:(service.c:177:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1769754739.887418:0:43811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3600/0x11e65116e9df2549 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd71 expref: 15 pid: 43811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754739.887442:0:43811:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100200/0x11e65116e9df255e lrc: 3/0,0 mode: PR/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd71 expref: 14 pid: 43811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754739.887451:0:43811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x17:0x0].0x0 (ffff90cb83cda500) 00010000:00010000:2.0:1769754739.887453:0:43811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1769754739.887456:0:43811:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff90cb92100200, rc 0) 00000100:00100000:2.0:1769754739.887994:0:43811:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb87860380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+15:50168:x1855720624217984:12345-192.168.202.6@tcp:101:bash.0 Request processed in 23142us (23274us total) trans 34359738535 rc 0/0 00000100:00100000:2.0:1769754739.888006:0:43811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 205 00000100:00100000:0.0:1769754739.898034:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:0.0:1769754739.898088:0:43451:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624218240 00000100:00100000:0.0:1769754739.898103:0:43451:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624218240 00000100:00100000:0.0:1769754739.898160:0:43451:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 57 00000100:00100000:0.0:1769754739.898170:0:43451:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb87188700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:0+-99:2368:x1855720624218240:12345-192.168.202.6@tcp:8:bash.0 00010000:00080000:0.0:1769754739.898211:0:43451:0:(ldlm_lib.c:1429:target_handle_connect()) lustre-OST0000: connection from 44b15f98-ec2b-44ce-9a84-650ae6cf82b2@192.168.202.6@tcp t0 exp 0000000000000000 cur 1722 last 0 00000020:00000080:0.0:1769754739.898259:0:43451:0:(genops.c:1402:class_connect()) connect: client 44b15f98-ec2b-44ce-9a84-650ae6cf82b2, cookie 0x11e65116e9df256c 00002000:00100000:0.0:1769754739.898277:0:43451:0:(ofd_obd.c:124:ofd_parse_connect_data()) lustre-OST0000: cli 44b15f98-ec2b-44ce-9a84-650ae6cf82b2/ffff90ca88729000 ocd_connect_flags: 0xa0465af2e3640038 ocd_version: 2113248 ocd_grant: 5111808 ocd_index: 0 ocd_group 0 00002000:00100000:0.0:1769754739.898310:0:43451:0:(ofd_obd.c:222:ofd_parse_connect_data()) lustre-OST0000: cli supports cksum type f7, return f7 00002000:00080000:0.0:1769754739.898416:0:43451:0:(ofd_obd.c:365:ofd_obd_connect()) lustre-OST0000: get connection from MDS 0 00000100:00080000:0.0:1769754739.898459:0:43451:0:(import.c:65:import_set_state_nolock()) ffff90ca8872b800 : changing import state from RECOVER to FULL 00010000:00010000:3.0:1769754739.898656:0:3304:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1800/0x11e65116e9df2557 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754739.898671:0:3304:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1800/0x11e65116e9df2557 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754739.898680:0:3304:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1800/0x11e65116e9df2557 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754739.898687:0:3304:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff90ca82ad1800 already unused, calling callback (ffffffffc2101550) 00010000:00010000:3.0:1769754739.898693:0:3304:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1800/0x11e65116e9df2557 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754739.898702:0:3304:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1800/0x11e65116e9df2557 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754739.898723:0:3304:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5d1d8ab3 (ffff90cb83c8f300) 00010000:00010000:3.0:1769754739.898727:0:3304:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769754739.898732:0:3304:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1800/0x11e65116e9df2557 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754739.898739:0:3304:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1800/0x11e65116e9df2557 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1769754739.901305:0:43451:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb87188700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+6:2368:x1855720624218240:12345-192.168.202.6@tcp:8:bash.0 Request processed in 3119us (3266us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754739.901325:0:43451:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 57 00000100:00100000:0.0:1769754739.908615:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:1.0:1769754739.908735:0:43451:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624218368 00000100:00100000:1.0:1769754739.908749:0:43451:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624218368 00000100:00100000:1.0:1769754739.908827:0:43451:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 58 00000100:00100000:1.0:1769754739.908837:0:43451:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb87189500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+6:0:x1855720624218368:12345-192.168.202.6@tcp:101:bash.0 00010000:00010000:1.0:1769754739.908858:0:43451:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1769754739.908890:0:43451:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ae526ef/0x11e65116e9df2573 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x106:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x2b94eb038208fd78 expref: -99 pid: 43451 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1769754739.908924:0:43451:0:(ofd_lvb.c:150:ofd_lvbo_init()) res: [0x240000400:0x106:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1769754739.908946:0:43451:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769754739.908949:0:43451:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769754739.908959:0:43451:0:(ldlm_resource.c:1714:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ae526ef/0x11e65116e9df2573 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x106:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd78 expref: 7 pid: 43451 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754739.908980:0:43451:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004ae526ef/0x11e65116e9df2573 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x106:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd78 expref: 7 pid: 43451 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754739.909011:0:43451:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff90ca887d7e00, rc 0) 00000100:00100000:1.0:1769754739.909077:0:43451:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb87189500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+7:0:x1855720624218368:12345-192.168.202.6@tcp:101:bash.0 Request processed in 242us (489us total) trans 0 rc 0/0 00000100:00100000:1.0:1769754739.909085:0:43451:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 58 00000100:00080000:0.0:1769754739.910170:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769754739.921066:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:2.0:1769754739.921552:0:42937:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624218496 00000100:00100000:2.0:1769754739.921568:0:42937:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624218496 00000100:00100000:2.0:1769754739.921652:0:42937:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 59 00000100:00100000:2.0:1769754739.921665:0:42937:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbfcb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+7:50168:x1855720624218496:12345-192.168.202.6@tcp:101:bash.0 00010000:00010000:2.0:1769754739.921698:0:42937:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769754739.921724:0:42937:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009fdbe807/0x11e65116e9df257a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x2b94eb038208fd7f expref: -99 pid: 42937 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754739.921778:0:42937:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000004ae526ef/0x11e65116e9df2573 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd78 expref: 8 pid: 43451 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754739.921810:0:42937:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769754739.921813:0:42937:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1769754739.921819:0:42937:0:(ldlm_resource.c:1714:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009fdbe807/0x11e65116e9df257a lrc: 4/0,0 mode: --/PW res: [0x240000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd7f expref: 8 pid: 42937 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754739.921864:0:42937:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000004ae526ef/0x11e65116e9df2573 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd78 expref: 8 pid: 43451 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754739.921879:0:42937:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000004ae526ef/0x11e65116e9df2573 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd78 expref: 8 pid: 43451 timeout: 1822 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1769754739.921895:0:42937:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cbb95e5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:42937:1855720644842240:192.168.202.6@tcp:104: 00000100:00100000:2.0:1769754739.922016:0:42937:0:(client.c:2690:ptlrpc_set_wait()) set ffff90cb915346c0 going to sleep for 11 seconds 00000040:00080000:0.0:1769754739.924606:0:42347:0:(llog.c:797:llog_process_thread()) index: 8, lh_last_idx: 8 synced_idx: 8 lgh_last_idx: 8 00000040:00080000:0.0:1769754739.924618:0:42347:0:(llog_cat.c:828:llog_cat_process_common()) processing log [0x1:0x32:0x0] at index 8 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1769754739.924680:0:42347:0:(llog.c:854:llog_process_thread()) stop processing plain [0x1:0x32:0x0] index 6 count 5 00000040:00080000:0.0:1769754739.924689:0:42347:0:(llog.c:854:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 9 count 2 00000100:00100000:0.0:1769754739.925198:0:42347:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb87188a80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:42347:1855720644842368:192.168.202.6@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1769754739.925350:0:42347:0:(client.c:2690:ptlrpc_set_wait()) set ffff90cb95fea000 going to sleep for 11 seconds 00000100:00100000:1.0:1769754739.928332:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:2.0:1769754739.929080:0:42937:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cbb95e5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:42937:1855720644842240:192.168.202.6@tcp:104: 00010000:00010000:2.0:1769754739.929135:0:42937:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009fdbe807/0x11e65116e9df257a lrc: 3/0,0 mode: --/PW res: [0x240000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd7f expref: 8 pid: 42937 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754739.929164:0:42937:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff90cb92101600, rc 0) 00000100:00100000:2.0:1769754739.929236:0:42937:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbfcb5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+8:50168:x1855720624218496:12345-192.168.202.6@tcp:101:bash.0 Request processed in 7575us (8186us total) trans 0 rc 0/0 00000100:00100000:2.0:1769754739.929246:0:42937:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 59 00000100:00100000:3.0:1769754739.937203:0:42300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624218624 00000100:00100000:3.0:1769754739.937224:0:42300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624218624 00010000:00010000:3.0:1769754739.937276:0:42300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000004ae526ef/0x11e65116e9df2573 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd78 expref: 8 pid: 43451 timeout: 1822 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1769754739.937343:0:42300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 38 00000100:00100000:3.0:1769754739.937354:0:42300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbfcb6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+8:2372:x1855720624218624:12345-192.168.202.6@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1769754739.937372:0:42300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1769754739.937401:0:42300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000004ae526ef/0x11e65116e9df2573 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x106:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd78 expref: 8 pid: 43451 timeout: 1822 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754739.937418:0:42300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000004ae526ef/0x11e65116e9df2573 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x106:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd78 expref: 8 pid: 43451 timeout: 1822 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754739.937448:0:42300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ae526ef/0x11e65116e9df2573 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x106:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd78 expref: 8 pid: 43451 timeout: 1822 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754739.937467:0:42300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769754739.937469:0:42300:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1769754739.937478:0:42300:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009fdbe807/0x11e65116e9df257a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd7f expref: 7 pid: 42937 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754739.937489:0:42300:0:(ldlm_resource.c:1714:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009fdbe807/0x11e65116e9df257a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd7f expref: 7 pid: 42937 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754739.937570:0:42300:0:(ldlm_lockd.c:1081:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000009fdbe807/0x11e65116e9df257a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd7f expref: 7 pid: 42937 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1769754739.937588:0:42300:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb87863100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:42300:1855720644842496:192.168.202.6@tcp:105: 00000100:00100000:0.0:1769754739.938143:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:0.0:1769754739.938204:0:42347:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb87188a80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:42347:1855720644842368:192.168.202.6@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1769754739.938412:0:43811:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624218752 00000100:00100000:0.0:1769754739.938418:0:43811:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624218752 00000100:00100000:0.0:1769754739.938459:0:43811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 206 00000100:00100000:0.0:1769754739.938466:0:43811:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb8718b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+14:50169:x1855720624218752:12345-192.168.202.6@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1769754739.938529:0:43811:0:(mdt_handler.c:7932:mdt_fid2path()) path get [0x200003ab1:0x14:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1769754739.938674:0:43811:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb8718b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+14:50169:x1855720624218752:12345-192.168.202.6@tcp:53:lhsmtool_posix.0 Request processed in 207us (532us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754739.938683:0:43811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 206 00000100:00100000:3.0:1769754739.941524:0:42300:0:(client.c:2690:ptlrpc_set_wait()) set ffff90cba065fe40 going to sleep for 11 seconds 00000100:00100000:0.0:1769754739.947421:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:3.0:1769754739.948178:0:42313:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624219008 00000100:00100000:3.0:1769754739.948192:0:42313:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624219008 00000100:00100000:3.0:1769754739.948272:0:42313:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 207 00000100:00100000:3.0:1769754739.948283:0:42313:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb87188a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+14:50169:x1855720624219008:12345-192.168.202.6@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1769754739.948309:0:42313:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769754739.948336:0:42313:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a2400/0x11e65116e9df2581 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x2b94eb038208fd86 expref: -99 pid: 42313 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754739.948468:0:42313:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a2a00/0x11e65116e9df2588 lrc: 3/1,0 mode: --/PR res: [0x200003ab1:0x14:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 42313 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754739.948485:0:42313:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a2a00/0x11e65116e9df2588 lrc: 3/1,0 mode: PR/PR res: [0x200003ab1:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 42313 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754739.948524:0:42313:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a2a00/0x11e65116e9df2588 lrc: 3/1,0 mode: PR/PR res: [0x200003ab1:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 42313 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1769754739.948540:0:42313:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200003ab1:0x14:0x0]: returning size 11 00010000:00010000:3.0:1769754739.948735:0:42313:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a2400/0x11e65116e9df2581 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd86 expref: 17 pid: 42313 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754739.948758:0:42313:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a2a00/0x11e65116e9df2588 lrc: 3/0,0 mode: PR/PR res: [0x200003ab1:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd86 expref: 16 pid: 42313 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1769754739.951207:0:42300:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb87863100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:42300:1855720644842496:192.168.202.6@tcp:105: 00010000:00010000:2.0:1769754739.951245:0:42300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769754739.951719:0:42300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbfcb6300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+7:2372:x1855720624218624:12345-192.168.202.6@tcp:103:ldlm_bl.0 Request processed in 14355us (23388us total) trans 0 rc 0/0 00000100:00100000:2.0:1769754739.951732:0:42300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 38 00010000:00010000:3.0:1769754739.953045:0:42313:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x14:0x0].0x0 (ffff90cb87176500) 00010000:00010000:3.0:1769754739.953050:0:42313:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1769754739.953055:0:42313:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff90ca820a2a00, rc 0) 00000001:00080000:2.0:1769754739.954348:0:5807:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-MDT0000: transno 34359738534 is committed 00000100:00100000:0.0:1769754739.955368:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:3.0:1769754739.956633:0:42313:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb87188a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+15:50169:x1855720624219008:12345-192.168.202.6@tcp:101:lhsmtool_posix.0 Request processed in 8337us (9212us total) trans 34359738536 rc 0/0 00000100:00100000:3.0:1769754739.957180:0:42313:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 207 00000100:00100000:1.0:1769754739.957311:0:42316:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624219264 00000100:00100000:1.0:1769754739.957323:0:42316:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624219264 00000100:00100000:2.0:1769754739.958298:0:42316:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 68 00000100:00100000:2.0:1769754739.958311:0:42316:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb87189f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+15:50168:x1855720624219264:12345-192.168.202.6@tcp:35:bash.0 00000100:00100000:2.0:1769754739.959336:0:42316:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb87189f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+16:50168:x1855720624219264:12345-192.168.202.6@tcp:35:bash.0 Request processed in 1022us (3971us total) trans 34359738537 rc 0/0 00000100:00100000:2.0:1769754739.959347:0:42316:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 68 00000100:00100000:0.0:1769754739.964048:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:3.0:1769754739.964257:0:42313:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624219520 00000100:00100000:3.0:1769754739.964272:0:42313:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624219520 00000100:00100000:3.0:1769754739.964357:0:42313:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 208 00000100:00100000:3.0:1769754739.964370:0:42313:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb8718aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+16:50169:x1855720624219520:12345-192.168.202.6@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1769754739.965141:0:42313:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb8718aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+16:50169:x1855720624219520:12345-192.168.202.6@tcp:57:lhsmtool_posix.0 Request processed in 769us (1104us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754739.965154:0:42313:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 208 00000040:00080000:1.0:1769754739.965884:0:42347:0:(llog.c:797:llog_process_thread()) index: 8, lh_last_idx: 8 synced_idx: 8 lgh_last_idx: 8 00000040:00080000:1.0:1769754739.965892:0:42347:0:(llog_cat.c:828:llog_cat_process_common()) processing log [0x1:0x32:0x0] at index 8 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1769754739.965930:0:42347:0:(llog.c:854:llog_process_thread()) stop processing plain [0x1:0x32:0x0] index 6 count 5 00000040:00080000:1.0:1769754739.965938:0:42347:0:(llog.c:854:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 9 count 2 00000100:00100000:0.0:1769754739.993245:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:1.0:1769754740.001689:0:42317:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624219776 00000100:00100000:1.0:1769754740.001724:0:42317:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624219776 00000100:00100000:1.0:1769754740.001843:0:42317:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 69 00000100:00100000:1.0:1769754740.001858:0:42317:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90ca83f87450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+16:50169:x1855720624219776:12345-192.168.202.6@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1769754740.002236:0:42317:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90ca83f87450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+17:50169:x1855720624219776:12345-192.168.202.6@tcp:35:lhsmtool_posix.0 Request processed in 381us (9006us total) trans 34359738538 rc 0/0 00000100:00100000:1.0:1769754740.002246:0:42317:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 69 00000100:00100000:3.0:1769754740.044804:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:2.0:1769754740.045026:0:43811:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624219904 00000100:00100000:2.0:1769754740.045040:0:43811:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624219904 00000100:00100000:2.0:1769754740.045111:0:43811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 209 00000100:00100000:2.0:1769754740.045124:0:43811:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb87863100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+17:50175:x1855720624219904:12345-192.168.202.6@tcp:101:lfs.0 00010000:00010000:2.0:1769754740.045154:0:43811:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1769754740.045186:0:43811:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df2596 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x2b94eb038208fd8d expref: -99 pid: 43811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754740.045274:0:43811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100a00/0x11e65116e9df259d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754740.045291:0:43811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100a00/0x11e65116e9df259d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754740.045309:0:43811:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100a00/0x11e65116e9df259d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1769754740.045386:0:43811:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 13824 00000004:00200000:2.0:1769754740.045390:0:43811:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 31 stripe count 1 MDT count 1 00000004:00010000:2.0:1769754740.045443:0:43811:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100a00/0x11e65116e9df259d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754740.045473:0:43811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92101400/0x11e65116e9df2596 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd8d expref: 19 pid: 43811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754740.045542:0:43811:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100a00/0x11e65116e9df259d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd8d expref: 18 pid: 43811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754740.045552:0:43811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff90cb83cdae00) 00010000:00010000:2.0:1769754740.045555:0:43811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1769754740.045560:0:43811:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff90cb92100a00, rc 0) 00000100:00100000:2.0:1769754740.045636:0:43811:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb87863100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+18:50175:x1855720624219904:12345-192.168.202.6@tcp:101:lfs.0 Request processed in 514us (847us total) trans 0 rc 0/0 00000100:00100000:2.0:1769754740.045653:0:43811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 209 00000100:00100000:3.0:1769754740.049177:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:0.0:1769754740.049328:0:42313:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624220160 00000100:00100000:0.0:1769754740.049341:0:42313:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624220160 00000100:00100000:0.0:1769754740.049439:0:42313:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 210 00000100:00100000:0.0:1769754740.049450:0:42313:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb87863480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+18:50175:x1855720624220160:12345-192.168.202.6@tcp:101:lfs.0 00010000:00010000:0.0:1769754740.049475:0:42313:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769754740.049531:0:42313:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1000/0x11e65116e9df25a4 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x2b94eb038208fd94 expref: -99 pid: 42313 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.049666:0:42313:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1c00/0x11e65116e9df25ab lrc: 3/1,0 mode: --/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 42313 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.049682:0:42313:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1c00/0x11e65116e9df25ab lrc: 3/1,0 mode: PR/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 42313 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.049707:0:42313:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1c00/0x11e65116e9df25ab lrc: 3/1,0 mode: PR/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 42313 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1769754740.049731:0:42313:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200003ab1:0x17:0x0]: returning size 19 00010000:00010000:0.0:1769754740.049890:0:42313:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1000/0x11e65116e9df25a4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd94 expref: 21 pid: 42313 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.049911:0:42313:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1c00/0x11e65116e9df25ab lrc: 3/0,0 mode: PR/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd94 expref: 20 pid: 42313 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.049922:0:42313:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x17:0x0].0x0 (ffff90cb83cda500) 00010000:00010000:0.0:1769754740.049925:0:42313:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1769754740.049929:0:42313:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff90ca82ad1c00, rc 0) 00000100:00100000:0.0:1769754740.050150:0:42313:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb87863480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+20:50175:x1855720624220160:12345-192.168.202.6@tcp:101:lfs.0 Request processed in 701us (986us total) trans 34359738539 rc 0/0 00000100:00100000:0.0:1769754740.050158:0:42313:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 210 00000100:00100000:1.0:1769754740.055629:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:1.0:1769754740.063123:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:2.0:1769754740.063183:0:43811:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624220416 00000100:00100000:2.0:1769754740.063198:0:43811:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624220416 00000100:00100000:2.0:1769754740.063240:0:42300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624220544 00000100:00100000:2.0:1769754740.063243:0:42300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624220544 00000100:00100000:2.0:1769754740.063326:0:42300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 16 00000100:00100000:2.0:1769754740.063340:0:42300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90ca88cc7450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+21:2371:x1855720624220544:12345-192.168.202.6@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769754740.063360:0:42300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769754740.063392:0:42300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1c00/0x11e65116e9df25ab lrc: 0/0,0 mode: --/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd94 expref: 21 pid: 42313 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754740.063418:0:42300:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x17:0x0].0x0 (ffff90cb83cda500) 00010000:00010000:2.0:1769754740.063424:0:42300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769754740.063487:0:42300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90ca88cc7450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+20:2371:x1855720624220544:12345-192.168.202.6@tcp:103:ldlm_bl.0 Request processed in 151us (379us total) trans 0 rc 0/0 00000100:00100000:2.0:1769754740.063534:0:42300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 16 00000100:00100000:3.0:1769754740.064130:0:43811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 211 00000100:00100000:3.0:1769754740.064140:0:43811:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbfcb5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+19:50175:x1855720624220416:12345-192.168.202.6@tcp:101:lfs.0 00010000:00010000:3.0:1769754740.064171:0:43811:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1769754740.064197:0:43811:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a2400/0x11e65116e9df25b9 lrc: 2/0,0 mode: --/CR res: [0x200003ab1:0x17:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x2b94eb038208fd9b expref: -99 pid: 43811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.064328:0:43811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3000/0x11e65116e9df25c0 lrc: 3/0,1 mode: --/PW res: [0x200003ab1:0x17:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.064347:0:43811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3000/0x11e65116e9df25c0 lrc: 3/0,1 mode: PW/PW res: [0x200003ab1:0x17:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.064359:0:43811:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3000/0x11e65116e9df25c0 lrc: 3/0,1 mode: PW/PW res: [0x200003ab1:0x17:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1769754740.064374:0:43811:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200003ab1:0x17:0x0]: returning size 19 00010000:00010000:3.0:1769754740.065470:0:43811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a2400/0x11e65116e9df25b9 lrc: 0/0,0 mode: --/CR res: [0x200003ab1:0x17:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd9b expref: 22 pid: 43811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.066347:0:43811:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3000/0x11e65116e9df25c0 lrc: 3/0,0 mode: PW/PW res: [0x200003ab1:0x17:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd9b expref: 21 pid: 43811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.066359:0:43811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x17:0x0].0x0 (ffff90cb83cda500) 00010000:00010000:3.0:1769754740.066363:0:43811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:3.0:1769754740.066367:0:43811:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff90ca820a3000, rc 0) 00000100:00100000:3.0:1769754740.066565:0:43811:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbfcb5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+21:50175:x1855720624220416:12345-192.168.202.6@tcp:101:lfs.0 Request processed in 2424us (10950us total) trans 34359738540 rc 0/0 00000100:00100000:3.0:1769754740.066576:0:43811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 211 00000100:00100000:1.0:1769754740.089000:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:2.0:1769754740.089100:0:42937:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624220672 00000100:00100000:2.0:1769754740.089114:0:42937:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624220672 00000100:00100000:2.0:1769754740.089191:0:42937:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 60 00000100:00100000:2.0:1769754740.089203:0:42937:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbfcb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+7:2370:x1855720624220672:12345-192.168.202.6@tcp:1:lfs.0 00010000:00010000:2.0:1769754740.089263:0:42937:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ffa604c/0x11e65116e9df25ce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 42937 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754740.089291:0:42937:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009fdbe807/0x11e65116e9df257a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd7f expref: 7 pid: 42937 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754740.089301:0:42937:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769754740.089303:0:42937:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 1 00010000:00010000:2.0:1769754740.089307:0:42937:0:(ldlm_resource.c:1714:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ffa604c/0x11e65116e9df25ce lrc: 4/0,1 mode: --/PW res: [0x240000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 42937 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754740.089353:0:42937:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009fdbe807/0x11e65116e9df257a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd7f expref: 7 pid: 42937 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754740.089369:0:42937:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009fdbe807/0x11e65116e9df257a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd7f expref: 7 pid: 42937 timeout: 1823 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1769754740.089386:0:42937:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cbb95e5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:42937:1855720644842624:192.168.202.6@tcp:104: 00000100:00100000:2.0:1769754740.089529:0:42937:0:(client.c:2690:ptlrpc_set_wait()) set ffff90cb915346c0 going to sleep for 11 seconds 00000100:00100000:2.0:1769754740.095425:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:3.0:1769754740.095532:0:42942:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624220800 00000100:00100000:3.0:1769754740.095545:0:42942:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624220800 00000100:00100000:0.0:1769754740.095709:0:42937:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cbb95e5f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:42937:1855720644842624:192.168.202.6@tcp:104: 00010000:00010000:0.0:1769754740.095746:0:42937:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000007ffa604c/0x11e65116e9df25ce lrc: 3/0,1 mode: --/PW res: [0x240000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 42937 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00100000:3.0:1769754740.095830:0:42942:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x106:0x0] (0->18) req@ffff90cbb95e7100 x1855720624220800/t0(0) o4->44b15f98-ec2b-44ce-9a84-650ae6cf82b2@192.168.202.6@tcp:21/0 lens 512/0 e 0 to 0 dl 1769754751 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lfs.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1769754740.095841:0:42942:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff90cbb95e7100 with x1855720624220800 ext(0->18) 00010000:00010000:3.0:1769754740.095849:0:42942:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000009fdbe807/0x11e65116e9df257a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd7f expref: 8 pid: 42937 timeout: 1823 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.095867:0:42942:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000009fdbe807/0x11e65116e9df257a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd7f expref: 8 pid: 42937 timeout: 1823 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1769754740.095875:0:42942:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff90cbb95e7100 00000100:00100000:3.0:1769754740.095908:0:42942:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 3 00000100:00100000:3.0:1769754740.095918:0:42942:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbb95e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+8:2372:x1855720624220800:12345-192.168.202.6@tcp:4:lfs.0 00000001:00100000:1.0:1769754740.198663:0:6576:0:(tgt_lastrcvd.c:532:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of 44b15f98-ec2b-44ce-9a84-650ae6cf82b2 00000001:00100000:1.0:1769754740.198701:0:6576:0:(tgt_lastrcvd.c:532:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of 44b15f98-ec2b-44ce-9a84-650ae6cf82b2 00000020:00000080:1.0:1769754740.198705:0:6576:0:(genops.c:904:class_export_put()) final put ffff90ca8872c800/44b15f98-ec2b-44ce-9a84-650ae6cf82b2 00000020:00000080:2.0:1769754740.198960:0:8:0:(genops.c:863:class_export_destroy()) destroying export ffff90ca8872c800/44b15f98-ec2b-44ce-9a84-650ae6cf82b2 for lustre-OST0000 00000020:00000080:2.0:1769754740.199100:0:8:0:(genops.c:1202:obd_zombie_import_free()) destroying import ffff90ca88729800 for lustre-OST0000 00000001:00080000:0.0:1769754740.292269:0:6576:0:(tgt_lastrcvd.c:950:tgt_cb_last_committed()) lustre-OST0000: transno 34359738383 is committed 00000100:00100000:1.0:1769754740.295919:0:42942:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbb95e7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+7:2372:x1855720624220800:12345-192.168.202.6@tcp:4:lfs.0 Request processed in 199984us (200489us total) trans 34359738383 rc 0/0 00000100:00100000:1.0:1769754740.295941:0:42942:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 3 00002000:00100000:1.0:1769754740.295958:0:42942:0:(ofd_dev.c:2706:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x106:0x0] (0->18) req@ffff90cbb95e7100 x1855720624220800/t34359738383(0) o4->44b15f98-ec2b-44ce-9a84-650ae6cf82b2@192.168.202.6@tcp:21/0 lens 512/448 e 0 to 0 dl 1769754751 ref 1 fl Complete:H/600/0 rc 0/0 job:'lfs.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1769754740.295970:0:42942:0:(ofd_dev.c:2565:ofd_prolong_extent_locks()) Prolong locks for req ffff90cbb95e7100 with x1855720624220800 ext(0->18) 00010000:00010000:1.0:1769754740.295984:0:42942:0:(ldlm_extent.c:773:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000009fdbe807/0x11e65116e9df257a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd7f expref: 7 pid: 42937 timeout: 1823 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754740.296008:0:42942:0:(ldlm_lockd.c:606:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000009fdbe807/0x11e65116e9df257a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd7f expref: 7 pid: 42937 timeout: 1823 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1769754740.296017:0:42942:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff90cbb95e7100 00000100:00100000:3.0:1769754740.300963:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:1.0:1769754740.301121:0:42299:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624220928 00000100:00100000:1.0:1769754740.301137:0:42299:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624220928 00010000:00010000:1.0:1769754740.301190:0:42299:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009fdbe807/0x11e65116e9df257a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd7f expref: 7 pid: 42937 timeout: 1823 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1769754740.301257:0:42299:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 39 00000100:00100000:1.0:1769754740.301269:0:42299:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb87860700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+7:2371:x1855720624220928:12345-192.168.202.6@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1769754740.301291:0:42299:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1769754740.301315:0:42299:0:(ofd_lvb.c:304:ofd_lvbo_update()) res: [0x240000400:0x106:0x0] updating lvb size from disk: 0 -> 19 00002000:00010000:1.0:1769754740.301319:0:42299:0:(ofd_lvb.c:311:ofd_lvbo_update()) res: [0x240000400:0x106:0x0] updating lvb mtime from disk: 0 -> 1769754740 00002000:00010000:1.0:1769754740.301322:0:42299:0:(ofd_lvb.c:317:ofd_lvbo_update()) res: [0x240000400:0x106:0x0] updating lvb atime from disk: 0 -> 1769754740 00002000:00010000:1.0:1769754740.301325:0:42299:0:(ofd_lvb.c:323:ofd_lvbo_update()) res: [0x240000400:0x106:0x0] updating lvb ctime from disk: 0 -> 1769754740 00002000:00010000:1.0:1769754740.301328:0:42299:0:(ofd_lvb.c:329:ofd_lvbo_update()) res: [0x240000400:0x106:0x0] updating lvb blocks from disk: 1 -> 11 00010000:00010000:1.0:1769754740.301343:0:42299:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009fdbe807/0x11e65116e9df257a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x106:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd7f expref: 7 pid: 42937 timeout: 1823 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754740.301360:0:42299:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009fdbe807/0x11e65116e9df257a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x106:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd7f expref: 7 pid: 42937 timeout: 1823 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754740.301395:0:42299:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009fdbe807/0x11e65116e9df257a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x106:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd7f expref: 7 pid: 42937 timeout: 1823 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754740.301418:0:42299:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769754740.301421:0:42299:0:(ldlm_extent.c:416:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1769754740.301427:0:42299:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000007ffa604c/0x11e65116e9df25ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 42937 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754740.301438:0:42299:0:(ldlm_resource.c:1714:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ffa604c/0x11e65116e9df25ce lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x106:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 42937 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1769754740.301479:0:42299:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1769754740.301875:0:42299:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb87860700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+6:2371:x1855720624220928:12345-192.168.202.6@tcp:103:ldlm_bl.0 Request processed in 606us (925us total) trans 0 rc 0/0 00000100:00100000:1.0:1769754740.301885:0:42299:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 39 00010000:00010000:0.0:1769754740.302408:0:42937:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000007ffa604c/0x11e65116e9df25ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x106:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 42937 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.302422:0:42937:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007ffa604c/0x11e65116e9df25ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x106:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 42937 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.302473:0:42937:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007ffa604c/0x11e65116e9df25ce lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x106:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 42937 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.302483:0:42937:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007ffa604c/0x11e65116e9df25ce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x106:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 42937 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.302524:0:42937:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ffa604c/0x11e65116e9df25ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x106:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 42937 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.302534:0:42937:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff90cb92101c00 already unused, calling callback (ffffffffc16a4360) 00010000:00010000:0.0:1769754740.302539:0:42937:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ffa604c/0x11e65116e9df25ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x106:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 42937 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.302549:0:42937:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ffa604c/0x11e65116e9df25ce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x106:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 42937 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.302563:0:42937:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ffa604c/0x11e65116e9df25ce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x106:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 42937 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.302571:0:42937:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ffa604c/0x11e65116e9df25ce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x106:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 42937 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1769754740.302681:0:42937:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbfcb6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+5:2370:x1855720624220672:12345-192.168.202.6@tcp:1:lfs.0 Request processed in 213479us (213695us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754740.302700:0:42937:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 60 00000100:00100000:3.0:1769754740.314753:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:0.0:1769754740.315055:0:42316:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624221184 00000100:00100000:0.0:1769754740.315070:0:42316:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624221184 00000100:00100000:0.0:1769754740.315141:0:42316:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 70 00000100:00100000:0.0:1769754740.315153:0:42316:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbd7e1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+21:50175:x1855720624221184:12345-192.168.202.6@tcp:35:lfs.0 00000004:00010000:0.0:1769754740.315256:0:42316:0:(mdt_open.c:1996:mdt_hsm_release()) ### [0x200003ab1:0x17:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3000/0x11e65116e9df25c0 lrc: 3/0,0 mode: PW/PW res: [0x200003ab1:0x17:0x0].0x0 bits 0x4/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd9b expref: 21 pid: 43811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1769754740.315630:0:42316:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 34359738541 00000001:00080000:0.0:1769754740.315697:0:42316:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 34359738542 00000001:00080000:0.0:1769754740.316336:0:42316:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 34359738543 00010000:00010000:0.0:1769754740.316405:0:42316:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0e00/0x11e65116e9df25d5 lrc: 3/0,1 mode: --/EX res: [0x200003ab1:0x17:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 42316 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.316427:0:42316:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100200/0x11e65116e9df255e lrc: 2/0,0 mode: PR/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd71 expref: 25 pid: 43811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.316437:0:42316:0:(ldlm_resource.c:1714:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0e00/0x11e65116e9df25d5 lrc: 4/0,1 mode: --/EX res: [0x200003ab1:0x17:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 42316 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.316488:0:42316:0:(ldlm_lockd.c:962:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100200/0x11e65116e9df255e lrc: 3/0,0 mode: PR/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd71 expref: 25 pid: 43811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.316960:0:42316:0:(ldlm_lockd.c:494:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100200/0x11e65116e9df255e lrc: 4/0,0 mode: PR/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd71 expref: 25 pid: 43811 timeout: 1823 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1769754740.316980:0:42316:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb87188000 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:42316:1855720644842752:192.168.202.6@tcp:104: 00000100:00100000:0.0:1769754740.317104:0:42316:0:(client.c:2690:ptlrpc_set_wait()) set ffff90cbb4851480 going to sleep for 11 seconds 00000100:00100000:0.0:1769754740.330796:0:42316:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb87188000 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:42316:1855720644842752:192.168.202.6@tcp:104: 00010000:00010000:0.0:1769754740.330843:0:42316:0:(ldlm_request.c:274:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0e00/0x11e65116e9df25d5 lrc: 3/0,1 mode: --/EX res: [0x200003ab1:0x17:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 42316 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1769754740.333200:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:2.0:1769754740.333547:0:42300:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624221312 00000100:00100000:2.0:1769754740.333560:0:42300:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624221312 00010000:00010000:2.0:1769754740.333622:0:42300:0:(ldlm_lockd.c:2675:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100200/0x11e65116e9df255e lrc: 4/0,0 mode: PR/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd71 expref: 26 pid: 43811 timeout: 1823 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1769754740.333685:0:42300:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 40 00000100:00100000:2.0:1769754740.333698:0:42300:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbd7e3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+26:2371:x1855720624221312:12345-192.168.202.6@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769754740.333719:0:42300:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769754740.333728:0:42300:0:(ldlm_lockd.c:1795:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100200/0x11e65116e9df255e lrc: 4/0,0 mode: PR/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd71 expref: 26 pid: 43811 timeout: 1823 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754740.333756:0:42300:0:(ldlm_lockd.c:567:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100200/0x11e65116e9df255e lrc: 3/0,0 mode: PR/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd71 expref: 26 pid: 43811 timeout: 1823 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754740.333786:0:42300:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90cb92100200/0x11e65116e9df255e lrc: 0/0,0 mode: --/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd71 expref: 26 pid: 43811 timeout: 1823 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754740.333805:0:42300:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x17:0x0].0x0 (ffff90cb83cda500) 00010000:00010000:2.0:1769754740.333811:0:42300:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0e00/0x11e65116e9df25d5 lrc: 3/0,1 mode: --/EX res: [0x200003ab1:0x17:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 42316 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754740.333824:0:42300:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0e00/0x11e65116e9df25d5 lrc: 3/0,1 mode: EX/EX res: [0x200003ab1:0x17:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 42316 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754740.333833:0:42300:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0e00/0x11e65116e9df25d5 lrc: 4/0,1 mode: EX/EX res: [0x200003ab1:0x17:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 42316 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754740.333862:0:42300:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769754740.333959:0:42300:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbd7e3100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+25:2371:x1855720624221312:12345-192.168.202.6@tcp:103:ldlm_bl.0 Request processed in 262us (772us total) trans 0 rc 0/0 00000100:00100000:2.0:1769754740.333969:0:42300:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 40 00010000:00010000:0.0:1769754740.334519:0:42316:0:(ldlm_request.c:186:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0e00/0x11e65116e9df25d5 lrc: 3/0,1 mode: EX/EX res: [0x200003ab1:0x17:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 42316 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.334530:0:42316:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0e00/0x11e65116e9df25d5 lrc: 3/0,1 mode: EX/EX res: [0x200003ab1:0x17:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 42316 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1769754740.334721:0:42316:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 34359738544 00000001:00080000:0.0:1769754740.335000:0:42316:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 34359738545 00000100:00100000:0.0:1769754740.335145:0:42316:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb87188000 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:42316:1855720644842880:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:0.0:1769754740.335224:0:42316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769754740.335273:0:42316:0:(client.c:2690:ptlrpc_set_wait()) set ffff90cb9567e300 going to sleep for 11 seconds 00000100:00100000:1.0:1769754740.335448:0:42319:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644842880 00000100:00100000:1.0:1769754740.335457:0:42319:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644842880 00000100:00100000:1.0:1769754740.335554:0:42319:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 12 00000100:00100000:1.0:1769754740.335564:0:42319:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb87189c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:42316:x1855720644842880:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:1.0:1769754740.335666:0:42319:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb87189c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:42316:x1855720644842880:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 103us (441us total) trans 0 rc 0/0 00000100:00100000:1.0:1769754740.335692:0:42319:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 12 00000100:00100000:0.0:1769754740.336201:0:42316:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb87188000 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:42316:1855720644842880:0@lo:1000:mdt_rdpg00_000.0 00000001:00080000:0.0:1769754740.336448:0:42316:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 34359738546 00000100:00100000:0.0:1769754740.336547:0:42316:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb87188000 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:42316:1855720644843008:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:0.0:1769754740.336604:0:42316:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769754740.336768:0:42316:0:(client.c:2690:ptlrpc_set_wait()) set ffff90cbb4851480 going to sleep for 11 seconds 00000100:00100000:3.0:1769754740.337489:0:42319:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644843008 00000100:00100000:3.0:1769754740.337534:0:42319:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644843008 00000100:00100000:3.0:1769754740.338218:0:42319:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 13 00000100:00100000:0.0:1769754740.339234:0:42319:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb8718ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:42316:x1855720644843008:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:0.0:1769754740.340198:0:42319:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb8718ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:42316:x1855720644843008:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 1962us (3589us total) trans 34359738384 rc 0/0 00000100:00100000:0.0:1769754740.340213:0:42319:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 13 00000100:00100000:2.0:1769754740.340399:0:42316:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb87188000 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:42316:1855720644843008:0@lo:1000:mdt_rdpg00_000.0 00000001:00080000:2.0:1769754740.341125:0:42316:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 34359738547 00010000:00010000:2.0:1769754740.341206:0:42316:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0e00/0x11e65116e9df25d5 lrc: 3/0,1 mode: EX/EX res: [0x200003ab1:0x17:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 42316 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754740.341220:0:42316:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0e00/0x11e65116e9df25d5 lrc: 2/0,0 mode: EX/EX res: [0x200003ab1:0x17:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 42316 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754740.341230:0:42316:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0e00/0x11e65116e9df25d5 lrc: 3/0,0 mode: EX/EX res: [0x200003ab1:0x17:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 42316 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754740.341238:0:42316:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff90ca82ad0e00 already unused, calling callback (ffffffffc2101550) 00010000:00010000:2.0:1769754740.341243:0:42316:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0e00/0x11e65116e9df25d5 lrc: 3/0,0 mode: EX/EX res: [0x200003ab1:0x17:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 42316 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754740.341252:0:42316:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0e00/0x11e65116e9df25d5 lrc: 4/0,0 mode: EX/EX res: [0x200003ab1:0x17:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 42316 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754740.341269:0:42316:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x17:0x0].0x0 (ffff90cb83cda500) 00010000:00010000:2.0:1769754740.341272:0:42316:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:2.0:1769754740.341279:0:42316:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0e00/0x11e65116e9df25d5 lrc: 2/0,0 mode: --/EX res: [0x200003ab1:0x17:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 42316 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754740.341285:0:42316:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0e00/0x11e65116e9df25d5 lrc: 0/0,0 mode: --/EX res: [0x200003ab1:0x17:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 42316 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1769754740.341976:0:42316:0:(mdd_object.c:3993:mdd_close()) Object [0x200003ab1:0x18:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:2.0:1769754740.342111:0:42316:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x2e:0x0].7, 64 off8664 00000004:00080000:2.0:1769754740.342144:0:42316:0:(osp_sync.c:1841:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1723329649813ns, next at 1724329647759ns, rc = 0 00000001:00080000:2.0:1769754740.342154:0:42316:0:(tgt_lastrcvd.c:2075:tgt_txn_stop_cb()) More than one transaction 34359738548 00010000:00010000:2.0:1769754740.342241:0:42316:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x17:0x0].0x0 (ffff90cb83cda500) 00010000:00010000:2.0:1769754740.342245:0:42316:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:2.0:1769754740.342255:0:42316:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3000/0x11e65116e9df25c0 lrc: 0/0,0 mode: --/PW res: [0x200003ab1:0x17:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fd9b expref: 29 pid: 43811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1769754740.342473:0:42316:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbd7e1500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+28:50175:x1855720624221184:12345-192.168.202.6@tcp:35:lfs.0 Request processed in 27319us (27742us total) trans 34359738549 rc 0/0 00000100:00100000:2.0:1769754740.342485:0:42316:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 70 00000020:00080000:2.0:1769754740.342518:0:42316:0:(tgt_main.c:782:tgt_ses_key_exit()) total 9 transactions per RPC 00000100:00100000:3.0:1769754740.346472:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:0.0:1769754740.346623:0:43811:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624221440 00000100:00100000:0.0:1769754740.346647:0:43811:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624221440 00000100:00100000:0.0:1769754740.346722:0:43811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 212 00000100:00100000:0.0:1769754740.346734:0:43811:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbbf87b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+28:50175:x1855720624221440:12345-192.168.202.6@tcp:101:lfs.0 00010000:00010000:0.0:1769754740.346794:0:43811:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769754740.346825:0:43811:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1600/0x11e65116e9df25dc lrc: 2/0,0 mode: --/CR res: [0x200003ab1:0x17:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x2b94eb038208fda2 expref: -99 pid: 43811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.346905:0:43811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1600/0x11e65116e9df25dc lrc: 3/0,0 mode: CR/CR res: [0x200003ab1:0x17:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fda2 expref: 29 pid: 43811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.346925:0:43811:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1600/0x11e65116e9df25dc lrc: 3/0,0 mode: CR/CR res: [0x200003ab1:0x17:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fda2 expref: 29 pid: 43811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.346947:0:43811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x17:0x0].0x0 (ffff90cb83c8fb00) 00010000:00010000:0.0:1769754740.346951:0:43811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1769754740.346956:0:43811:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff90ca82ad1600, rc 0) 00000100:00100000:0.0:1769754740.347025:0:43811:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbbf87b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+29:50175:x1855720624221440:12345-192.168.202.6@tcp:101:lfs.0 Request processed in 293us (567us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754740.347033:0:43811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 212 00000100:00100000:3.0:1769754740.351054:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:2.0:1769754740.351248:0:42316:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624221696 00000100:00100000:2.0:1769754740.351263:0:42316:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624221696 00000100:00100000:2.0:1769754740.351347:0:42316:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 71 00000100:00100000:2.0:1769754740.351360:0:42316:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbbf84700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+29:50175:x1855720624221696:12345-192.168.202.6@tcp:35:lfs.0 00000100:00100000:2.0:1769754740.351746:0:42316:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbbf84700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+30:50175:x1855720624221696:12345-192.168.202.6@tcp:35:lfs.0 Request processed in 379us (697us total) trans 34359738550 rc 0/0 00000100:00100000:2.0:1769754740.351756:0:42316:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 71 00010000:00010000:2.0:1769754740.437130:0:42301:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1769754740.474678:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:3.0:1769754740.475003:0:42314:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624221824 00000100:00100000:3.0:1769754740.475018:0:42314:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624221824 00000100:00100000:3.0:1769754740.475085:0:42314:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 213 00000100:00100000:3.0:1769754740.475096:0:42314:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbfcb4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+30:50185:x1855720624221824:12345-192.168.202.6@tcp:34:lfs.0 00010000:00010000:3.0:1769754740.475202:0:42314:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a2e00/0x11e65116e9df25e3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.475221:0:42314:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a2e00/0x11e65116e9df25e3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.475235:0:42314:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a2e00/0x11e65116e9df25e3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.475250:0:42314:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a2400/0x11e65116e9df25ea lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.475257:0:42314:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a2400/0x11e65116e9df25ea lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.475263:0:42314:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a2400/0x11e65116e9df25ea lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.475402:0:42314:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3200/0x11e65116e9df25f1 lrc: 3/1,0 mode: --/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.475434:0:42314:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3200/0x11e65116e9df25f1 lrc: 3/1,0 mode: PR/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.475442:0:42314:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3200/0x11e65116e9df25f1 lrc: 3/1,0 mode: PR/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1769754740.475714:0:42314:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3200/0x11e65116e9df25f1 lrc: 3/1,0 mode: PR/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.475732:0:42314:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a2e00/0x11e65116e9df25e3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.475749:0:42314:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a2e00/0x11e65116e9df25e3 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.475757:0:42314:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a2e00/0x11e65116e9df25e3 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.475764:0:42314:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff90ca820a2e00 already unused, calling callback (ffffffffc2101550) 00010000:00010000:3.0:1769754740.475770:0:42314:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a2e00/0x11e65116e9df25e3 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.475779:0:42314:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a2e00/0x11e65116e9df25e3 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.475792:0:42314:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff90cb83cdae00) 00010000:00010000:3.0:1769754740.475795:0:42314:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769754740.475800:0:42314:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a2e00/0x11e65116e9df25e3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.475806:0:42314:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a2e00/0x11e65116e9df25e3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.475830:0:42314:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a2400/0x11e65116e9df25ea lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.475836:0:42314:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a2400/0x11e65116e9df25ea lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.475843:0:42314:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a2400/0x11e65116e9df25ea lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.475848:0:42314:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff90ca820a2400 already unused, calling callback (ffffffffc2101550) 00010000:00010000:3.0:1769754740.475851:0:42314:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a2400/0x11e65116e9df25ea lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.475858:0:42314:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a2400/0x11e65116e9df25ea lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.475866:0:42314:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5d1d8ab3 (ffff90cb87176900) 00010000:00010000:3.0:1769754740.475869:0:42314:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769754740.475872:0:42314:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a2400/0x11e65116e9df25ea lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.475877:0:42314:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a2400/0x11e65116e9df25ea lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.475894:0:42314:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3200/0x11e65116e9df25f1 lrc: 3/1,0 mode: PR/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.475900:0:42314:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3200/0x11e65116e9df25f1 lrc: 2/0,0 mode: PR/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.475906:0:42314:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3200/0x11e65116e9df25f1 lrc: 3/0,0 mode: PR/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.475917:0:42314:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff90ca820a3200 already unused, calling callback (ffffffffc2101550) 00010000:00010000:3.0:1769754740.475920:0:42314:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3200/0x11e65116e9df25f1 lrc: 3/0,0 mode: PR/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.475927:0:42314:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3200/0x11e65116e9df25f1 lrc: 4/0,0 mode: PR/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.475934:0:42314:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x17:0x0].0x0 (ffff90cb83c8fb00) 00010000:00010000:3.0:1769754740.475936:0:42314:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1769754740.475939:0:42314:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3200/0x11e65116e9df25f1 lrc: 2/0,0 mode: --/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1769754740.475945:0:42314:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca820a3200/0x11e65116e9df25f1 lrc: 0/0,0 mode: --/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 42314 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1769754740.476034:0:42314:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbfcb4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+30:50185:x1855720624221824:12345-192.168.202.6@tcp:34:lfs.0 Request processed in 937us (1368us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754740.476043:0:42314:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 213 00000100:00100000:3.0:1769754740.575820:0:3312:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:0.0:1769754740.575951:0:43811:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624221952 00000100:00100000:0.0:1769754740.575967:0:43811:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624221952 00000100:00100000:0.0:1769754740.576082:0:43811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 214 00000100:00100000:0.0:1769754740.576095:0:43811:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbbf84000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+30:50191:x1855720624221952:12345-192.168.202.6@tcp:101:lfs.0 00010000:00010000:0.0:1769754740.576125:0:43811:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769754740.576151:0:43811:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0a00/0x11e65116e9df25f8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x2b94eb038208fda9 expref: -99 pid: 43811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.576239:0:43811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0600/0x11e65116e9df25ff lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.576257:0:43811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0600/0x11e65116e9df25ff lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.576291:0:43811:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0600/0x11e65116e9df25ff lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.576324:0:43811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1400/0x11e65116e9df2606 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.576334:0:43811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1400/0x11e65116e9df2606 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.576341:0:43811:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1400/0x11e65116e9df2606 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.576468:0:43811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0c00/0x11e65116e9df260d lrc: 3/1,0 mode: --/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.576477:0:43811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0c00/0x11e65116e9df260d lrc: 3/1,0 mode: PR/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.576484:0:43811:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0c00/0x11e65116e9df260d lrc: 3/1,0 mode: PR/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1769754740.576572:0:43811:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200003ab1:0x17:0x0]: returning size 19 00000004:00200000:0.0:1769754740.576578:0:43811:0:(mdt_handler.c:1577:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1769754740.576600:0:43811:0:(mdt_handler.c:2497:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0c00/0x11e65116e9df260d lrc: 3/1,0 mode: PR/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.576612:0:43811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0c00/0x11e65116e9df260d lrc: 3/1,0 mode: PR/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.576622:0:43811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0600/0x11e65116e9df25ff lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.576649:0:43811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0600/0x11e65116e9df25ff lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.576656:0:43811:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0600/0x11e65116e9df25ff lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.576663:0:43811:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff90ca82ad0600 already unused, calling callback (ffffffffc2101550) 00010000:00010000:0.0:1769754740.576667:0:43811:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0600/0x11e65116e9df25ff lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.576675:0:43811:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0600/0x11e65116e9df25ff lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.576689:0:43811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff90cb83cdae00) 00010000:00010000:0.0:1769754740.576692:0:43811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769754740.576698:0:43811:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0600/0x11e65116e9df25ff lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.576704:0:43811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0600/0x11e65116e9df25ff lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.576721:0:43811:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1400/0x11e65116e9df2606 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.576728:0:43811:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1400/0x11e65116e9df2606 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.576734:0:43811:0:(ldlm_lockd.c:1906:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1400/0x11e65116e9df2606 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.576740:0:43811:0:(ldlm_lockd.c:1920:ldlm_handle_bl_callback()) Lock ffff90ca82ad1400 already unused, calling callback (ffffffffc2101550) 00010000:00010000:0.0:1769754740.576750:0:43811:0:(ldlm_request.c:355:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1400/0x11e65116e9df2606 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.576757:0:43811:0:(ldlm_request.c:1310:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1400/0x11e65116e9df2606 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.576766:0:43811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5d1d8ab3 (ffff90cb83c8fa00) 00010000:00010000:0.0:1769754740.576769:0:43811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1769754740.576772:0:43811:0:(ldlm_lockd.c:1932:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1400/0x11e65116e9df2606 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.576778:0:43811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1400/0x11e65116e9df2606 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5d1d8ab3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.576816:0:43811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0a00/0x11e65116e9df25f8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fda9 expref: 32 pid: 43811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.576832:0:43811:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0c00/0x11e65116e9df260d lrc: 3/0,0 mode: PR/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fda9 expref: 31 pid: 43811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.576839:0:43811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x17:0x0].0x0 (ffff90cb83c8fb00) 00010000:00010000:0.0:1769754740.576842:0:43811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1769754740.576844:0:43811:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff90ca82ad0c00, rc 0) 00000100:00100000:0.0:1769754740.576927:0:43811:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbbf84000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+31:50191:x1855720624221952:12345-192.168.202.6@tcp:101:lfs.0 Request processed in 834us (1122us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754740.576936:0:43811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 214 00000100:00100000:1.0:1769754740.582104:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:0.0:1769754740.582611:0:43811:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624222208 00000100:00100000:0.0:1769754740.582625:0:43811:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624222208 00000100:00100000:0.0:1769754740.582711:0:43811:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 215 00000100:00100000:0.0:1769754740.582722:0:43811:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbfcb6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+31:50191:x1855720624222208:12345-192.168.202.6@tcp:101:lfs.0 00010000:00010000:0.0:1769754740.582766:0:43811:0:(ldlm_lockd.c:1264:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1769754740.582793:0:43811:0:(ldlm_lockd.c:1342:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0a00/0x11e65116e9df2614 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x2b94eb038208fdb0 expref: -99 pid: 43811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.582937:0:43811:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1400/0x11e65116e9df261b lrc: 3/1,0 mode: --/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.582956:0:43811:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1400/0x11e65116e9df261b lrc: 3/1,0 mode: PR/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.582970:0:43811:0:(ldlm_request.c:514:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1400/0x11e65116e9df261b lrc: 3/1,0 mode: PR/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 43811 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1769754740.582983:0:43811:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200003ab1:0x17:0x0]: returning size 19 00010000:00010000:0.0:1769754740.583144:0:43811:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad0a00/0x11e65116e9df2614 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fdb0 expref: 34 pid: 43811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.583168:0:43811:0:(ldlm_lockd.c:1512:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1400/0x11e65116e9df261b lrc: 3/0,0 mode: PR/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fdb0 expref: 33 pid: 43811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1769754740.583177:0:43811:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x17:0x0].0x0 (ffff90cb83c8fb00) 00010000:00010000:0.0:1769754740.583180:0:43811:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1769754740.583184:0:43811:0:(ldlm_lockd.c:1591:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff90ca82ad1400, rc 0) 00000100:00100000:0.0:1769754740.583375:0:43811:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbfcb6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+33:50191:x1855720624222208:12345-192.168.202.6@tcp:101:lfs.0 Request processed in 654us (1285us total) trans 34359738551 rc 0/0 00000100:00100000:0.0:1769754740.583383:0:43811:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 215 00000100:00100000:1.0:1769754740.589384:0:3313:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:0.0:1769754740.591486:0:42317:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624222464 00000100:00100000:1.0:1769754740.593067:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:2.0:1769754740.593192:0:42299:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624222592 00000100:00100000:2.0:1769754740.593207:0:42299:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624222592 00000100:00100000:2.0:1769754740.593298:0:42299:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 17 00000100:00100000:2.0:1769754740.593310:0:42299:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbc1449500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+33:2371:x1855720624222592:12345-192.168.202.6@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1769754740.593332:0:42299:0:(ldlm_lockd.c:1746:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1769754740.593365:0:42299:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff90ca82ad1400/0x11e65116e9df261b lrc: 0/0,0 mode: --/PR res: [0x200003ab1:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.202.6@tcp remote: 0x2b94eb038208fdb0 expref: 33 pid: 43811 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1769754740.593408:0:42299:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200003ab1:0x17:0x0].0x0 (ffff90cb83c8fb00) 00010000:00010000:2.0:1769754740.593414:0:42299:0:(ldlm_lockd.c:1807:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1769754740.593484:0:42299:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbc1449500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+32:2371:x1855720624222592:12345-192.168.202.6@tcp:103:ldlm_bl.0 Request processed in 176us (802us total) trans 0 rc 0/0 00000100:00100000:2.0:1769754740.593536:0:42299:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 17 00000100:00100000:0.0:1769754740.596252:0:42317:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624222464 00000100:00100000:0.0:1769754740.596367:0:42317:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 72 00000100:00100000:0.0:1769754740.596404:0:42317:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbbfcb6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+32:50191:x1855720624222464:12345-192.168.202.6@tcp:35:lfs.0 00000100:00100000:0.0:1769754740.596862:0:42317:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbbfcb6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+33:50191:x1855720624222464:12345-192.168.202.6@tcp:35:lfs.0 Request processed in 462us (7493us total) trans 34359738552 rc 0/0 00000100:00100000:0.0:1769754740.596872:0:42317:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 72 00000100:00100000:2.0:1769754740.884075:0:3319:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb8718b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3319:1855720644843136:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1769754740.884189:0:3319:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1769754740.884370:0:42939:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644843136 00000100:00100000:3.0:1769754740.884382:0:42939:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644843136 00000100:00100000:3.0:1769754740.884456:0:42939:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 45 00000100:00100000:3.0:1769754740.884480:0:42939:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cbb95e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1855720644843136:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1769754740.884547:0:42939:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 49348608 pending 0 free 3774873600 avail 3772776448 00000100:00100000:3.0:1769754740.884953:0:42939:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cbb95e5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3319:x1855720644843136:12345-0@lo:13:osp-pre-1-0.0 Request processed in 483us (760us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754740.884962:0:42939:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 45 00000100:00100000:2.0:1769754740.885117:0:3319:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb8718b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3319:1855720644843136:0@lo:13:osp-pre-1-0.0 00000004:00080000:0.0:1769754741.396795:0:42336:0:(osp_sync.c:1841:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 1724384297869ns, next at 1725384294371ns, rc = 0 00000100:00100000:3.0:1769754741.396844:0:3321:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1769754741.396895:0:3321:0:(client.c:1903:ptlrpc_send_new_req()) Sending RPC req@ffff90cb8718bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3321:1855720644843264:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1769754741.396987:0:3321:0:(events.c:340:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1769754741.397128:0:42940:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720644843264 00000100:00100000:0.0:1769754741.397139:0:42940:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720644843264 00000100:00100000:0.0:1769754741.397211:0:42940:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 46 00000100:00100000:0.0:1769754741.397220:0:42940:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb98352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3321:x1855720644843264:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1769754741.397274:0:42940:0:(ofd_obd.c:714:ofd_statfs()) blocks cached 0 granted 45940736 pending 0 free 3695181824 avail 3693084672 00000100:00100000:0.0:1769754741.397324:0:42940:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb98352300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3321:x1855720644843264:12345-0@lo:13:osp-pre-0-0.0 Request processed in 106us (337us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754741.397332:0:42940:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 46 00000100:00100000:3.0:1769754741.397422:0:3321:0:(client.c:2389:ptlrpc_check_set()) Completed RPC req@ffff90cb8718bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3321:1855720644843264:0@lo:13:osp-pre-0-0.0 00000100:00080000:2.0:1769754741.397870:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1769754741.459898:0:42302:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1769754742.401161:0:46205:0:(debug.c:665:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity-pcc test_17: @@@@@@ FAIL: /mnt/lustre/f17.sanity-pcc expected pcc state: none, but got: readwrite 00010000:00010000:2.0:1769754742.485547:0:43453:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1769754742.592369:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:0.0:1769754742.594854:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:0.0:1769754742.594914:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:0.0:1769754742.594936:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:0.0:1769754742.594969:0:3311:0:(events.c:340:request_in_callback()) peer: 12345-192.168.202.6@tcp (source: 12345-192.168.202.6@tcp) 00000100:00100000:2.0:1769754742.595176:0:42308:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624222720 00000100:00100000:2.0:1769754742.595190:0:42308:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624222720 00000100:00100000:2.0:1769754742.595257:0:42308:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 154 00000100:00100000:2.0:1769754742.595269:0:42308:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90ca83cbd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:cd1d8275-6a1a-49a3-b443-af2f7055b864+12:2369:x1855720624222720:12345-192.168.202.6@tcp:400:kworker.0 00000100:00100000:0.0:1769754742.595365:0:42937:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624222976 00000100:00100000:0.0:1769754742.595370:0:42937:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624222976 00000100:00100000:0.0:1769754742.595424:0:42937:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624223104 00000100:00100000:0.0:1769754742.595428:0:42937:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624223104 00000100:00100000:0.0:1769754742.595452:0:42937:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 80 00000100:00100000:0.0:1769754742.595460:0:42937:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb87188380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+5:2369:x1855720624222976:12345-192.168.202.6@tcp:400:kworker.0 00000100:00100000:1.0:1769754742.596327:0:43451:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 81 00000100:00100000:1.0:1769754742.596341:0:43451:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb87188e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+6:2369:x1855720624223104:12345-192.168.202.6@tcp:400:kworker.0 00000100:00100000:3.0:1769754742.598965:0:42314:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624223232 00000100:00100000:3.0:1769754742.603546:0:42314:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624223232 00000100:00100000:3.0:1769754742.603682:0:42314:0:(service.c:2279:ptlrpc_server_handle_req_in()) unwrap req x1855720624222848 00000100:00100000:3.0:1769754742.603687:0:42314:0:(service.c:2344:ptlrpc_server_handle_req_in()) got req x1855720624222848 00000100:00100000:3.0:1769754742.603728:0:42314:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 104 00000100:00100000:3.0:1769754742.603741:0:42314:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb8718bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:99ce4681-381f-407c-90f1-2ef98266ced9+5:2369:x1855720624223232:12345-192.168.202.6@tcp:400:kworker.0 00000100:00100000:3.0:1769754742.603858:0:42314:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb8718bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:99ce4681-381f-407c-90f1-2ef98266ced9+5:2369:x1855720624223232:12345-192.168.202.6@tcp:400:kworker.0 Request processed in 119us (8942us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754742.603868:0:42314:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 104 00000100:00100000:3.0:1769754742.603891:0:42314:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.6@tcp, seq: 105 00000100:00100000:3.0:1769754742.603898:0:42314:0:(service.c:2530:ptlrpc_server_handle_request()) Handling RPC req@ffff90cb8718b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+33:2369:x1855720624222848:12345-192.168.202.6@tcp:400:kworker.0 00000100:00100000:3.0:1769754742.603933:0:42314:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb8718b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+33:2369:x1855720624222848:12345-192.168.202.6@tcp:400:kworker.0 Request processed in 36us (8996us total) trans 0 rc 0/0 00000100:00100000:3.0:1769754742.603940:0:42314:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 105 00000100:00100000:2.0:1769754742.604570:0:42308:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90ca83cbd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:cd1d8275-6a1a-49a3-b443-af2f7055b864+12:2369:x1855720624222720:12345-192.168.202.6@tcp:400:kworker.0 Request processed in 9297us (12208us total) trans 0 rc 0/0 00000100:00100000:2.0:1769754742.604582:0:42308:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 154 00000100:00100000:0.0:1769754742.605613:0:42937:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb87188380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+5:2369:x1855720624222976:12345-192.168.202.6@tcp:400:kworker.0 Request processed in 10145us (10764us total) trans 0 rc 0/0 00000100:00100000:0.0:1769754742.605663:0:42937:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 80 00000100:00080000:0.0:1769754742.605710:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1769754742.605716:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1769754742.605720:0:42309:0:(pinger.c:498:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1769754742.608192:0:43451:0:(service.c:2588:ptlrpc_server_handle_request()) Handled RPC req@ffff90cb87188e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:44b15f98-ec2b-44ce-9a84-650ae6cf82b2+6:2369:x1855720624223104:12345-192.168.202.6@tcp:400:kworker.0 Request processed in 11836us (13203us total) trans 0 rc 0/0 00000100:00100000:1.0:1769754742.608211:0:43451:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.6@tcp, seq: 81 00010000:00010000:2.0:1769754743.511047:0:42301:0:(ldlm_lockd.c:2796:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 1980 lines, 1980 kept, 0 dropped, 0 bad.